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students in nidi worth-while experiences. ReiJr^rij 
am ion to thoee itudenc ecuvuiei and organixv' 
instructional nature* we observe on moat^m 
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rruDurr AcnrmM 

droa th. aw t» l» bmirt d«i « lw ort <h 

«ty of speech, ud sometunro extensive shynm* which m»y Kem 
from myriad cau«« Yet "iany nicceaifu! piruc.panu and 
mo leaden may whibit theie same characteristics What- 
ever the Inhibitors of desired participation, the prevalence of 
non-participation indicates a serious problem for many aw- 
not only from the ewndporot of mental hypene but. 
equally so from the pomt of the umvemty’i troponiibiUry for 
UiIium youth in community and ntweiwhip rwponubilitiei 
- Mid even in the everyday living and working with othem with¬ 
out (fiction. If publicity alone does not produce participation 
' of those who deairt and need euch experiences, then what can 
be dbno to guide student* whh reepo* to thU phwe of perianal 
development Parenthetically, it a no amall catk re identify 
by name i»r! person thoee who do not partiopste, Since thqj 
are not pbyecatly present at etudoit event* and «[ain, s P Ka *‘ 
m«M> must be employed to personalize and individualize wo* 

Numerous attempt* have been nuda In the Unfvemty of 
Minnesota to evolve % program which would Facilitate the in¬ 
duction of students into activities of interest to then. In 
duK attempts there was no thought of hlgh-p«minnf afl itu- 
denta without regard! to penonil praferences of some for nor¬ 
mally reametad p.rtWp.tion. Rather it was thought that 
publicity wgardmg the wide range of avaiable opportunities 
’ and the Increasing ol this range would ipao facto attract »U 
itudentt who dcarad to 0 artidpate. Otir eapenwco did not 
eoropLetely Verify nch an assumption, Even the development 
pf s comprehpiaiyd and low-crot program of recreation in the 
Coffman Memorial Student Union, while unusually aue- 
cmful hi Jnrttaslng psrtkipadoti In recreational affaire, ha* 
fcot solved the problems ol an appreciable number of student* 
vriio asp run fl desire to join organization* but who do not 
letoond tn man invitations of the publicity type. 

A brief review of other program development* m this Uw- 
tmiiy will «W 0 « background for the most recent innovation. 
Tn 1930 student leaden petitioned the late Preaidenc Coffman 
to appoint a committee, with the wnter aa chairman, to atudy 


in a comprehensive way the social adjustment problem* or 
student*. Prior to this time an informal group of students and 
staff memberi had constituted itself a committee to coordinate 
the many duplicating and competing social programs, chiefly 
those of a duico nature. Under the leadership of the Univer¬ 
sity YMCA and YWCA a special “Fortnightly” program was 
developed, designed to induct the socially umid and awkward 
into college recreational programs, A balanced program qf 
non-dancing games and beginner's lessons proved to be very 
effective until its popularity attracted those who were already 
well versed rn social savoir faire, thus crowding out those leu 
adept, Interesting vanety In the standardized pattern or 
couple-dancing was introduced by the department of physical; 
education by means of square dances, outing activities, and' 
intra-mural sports, 

The eominirtnV survey stimulated an eapinrion of thp r f 
social-recreational program of the Student Union and led to 
the appointment of a Social Coordinating Committee which 
continues its attempts to plan n community program for tfijt 
University as a whole. TWs committee, working with "the , 
Calendar Committee of die All-Urn verity Student Council^ 
his achieved some success with respect to the problem of coriSI 
filets in functions, programs and schedules of Social activities, ' 

At President Coffman’* suggestion the Assistant Dean ol ' 
Men developed a Leadership Course to coordinate and improve' * 
the efforts of student leaden in their conduct ond leadership of ij 
organizational affaire. Little effect was noticeable with respect , 
to increasing social participation but this group did achieve * 
aome success through in Social Etiquette or Skills Courie. 

The YMCA and YWCA achieved notable success in the 
development of small chibs and discussion group! in which 
social adjustment was facilitated through intimacy of contacts’ 
and concentration of program events upon specific common^ 
interests. The YWCA groups are impermanent discussion 
groups while the YMCA groups are fraternal buc non-Greek, 
latter clubs of B continuing type. 

The above brief review indicates some of the expanded 
recroitidhal facilities Failure to participate was noc caused 
by scarcity of opportunities But recreation is only one 6 fi the 


' , STUDaNT ACTIVITIES < 

many types of social adjustment opportunities* In » normal 

- peace-time yearover three hundred student organizations spon¬ 
sor many types of activities in addition to the recreational 
program conducted by each organization largely lor in own 
members Th«a» three hundred oigimzatiooa cover the full 
range found On any campus, and provide nch opportunities tor 
social adjustment, personal development and atuenahip train- 

" mg, The Intensity of appeal, issued by these oiganitationi 
» for student workers and participants and the relatively smill 
^ attendance achieved at thmr affaire probably indicate both the 

- fceffectivoneet of the advertising methods need and the Inertia 
or timidity of non-pattiapznng etudeotz. Each year during 
Freshman Week new students ate harangued and cajoled witn 
i«ard to "corning out" for activities “after getting your study 

habit* developed " , 

Clearly lUidmta at Minne.au ire presented with oppor- 
■ (unifies and rather full information about them Should one 
, conclude that all thorn who derire to participate respond to 
• throe maia appeal* and that those who remain away'do ao 
" Sreuib deilre nr are sarufacronly adjusted in aocial rdebon- 
riW The ifi)2 survey of parricipitian (to hive bexn re- 
^ toSrtid ip 1942V showed otherwise Many mm-participants 
Vtpotud txekqf knowledge oF how in begin or indicated hesi- 
- fancy in reding die flret step—perb ape maw advertising meth¬ 
ods havorea-cMd tii«Tr nUpbrnum usefulness and must be«>- 
‘pitmtnttd by ofhef -mot® bi'diyiJu.liaed methods, Pernajw 
1 Wped litrollel&td heed the experience of gmup workers in 
ajmraiinfty sgenoic* tfith xtgpci to the lpe«*l need for pe> 

^ bnd imJvnduSnud induction of new members into 

imbp-lif*,' P«wps we pi |finne*Ota will find that, 
._.ew hare beep idvtd elw^hait, but an epmstve 
qf the literature and pefaonal hjnetvation* jndicaiee 
.lownpjre-moicate . 
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Aa of July 1,1943, certain functions related to the lUper- 
vwion of student orgicitationa and activities were regrouped 
administratively In a new Student Acrivitifa Bureau within the 
Office of the Dean, of Stpdenu, The following functions were 
assigned lothABuresu'a.itsff 
1 Ad 
Sit 

S 


3. Cooperation with faculty adviiere « iteaenc 
development in atudme prnniiational leaden 
sundiag of group and in dividual 
end interests of the man. 

4 Financial sunerviilon of 
5. CUunseilng individual students regarding, die ,cp 
progress in extncunicular acnviHa, or^ta^ao^w; 

The Bureau** staff is composed pf bod) mefi-aod women an 
includes two graduate auiitjanta Thu staff will deal with the! 
organizational and activity affaire of both men and women 
thereby differing markedly' from the situation on many car' 
puses where separate offldro often make for difficulties wi 
respect to affaire organized for both men and women. In am ^ 
a middle western state university as Minnesota, the nunibgl 
or joint aqtretdea' has markedly increased In die put twi 
decades. Separate stpdent organizations for women will, how 
ever, he continued and the special interests and problems 
women will not be ignored. 

■ Thf fln£ ajtd fourth functions listed above need no drocn 
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tioa in thia paper The second function constitute* the Uni¬ 
versity'* equivalent of community group work in * social 
agency The naff member* aerve both 11 group-work auper- 
viaon and program adviser*, and atudent leaden and faculty 
adviser* aerve aa volunteer group leaden The Bureau’* staff 
serves m advuory and lupemaory capacities and not u admin¬ 
istrator* except in those cases when au organization** program 
n of such a nature aj to jeopardise the University*! public 
relationships. Czpenenee indicate* dearly that rich instances 
occur infrequently when the relationships between student 
leaden and staff memben are characterized by the term ad¬ 
visory as opposed to administrative. The staff serve* a* a 
resource of suggestions regarding new program and event* 
when atudent leaden and faculty advisers seek aaajjitance. 
Without such new ideas, organizational programs sometime* 
tend to become monotonously itereotyped and faff to attract 
student participants. A further ph«e of program advising 
involves the coordination and eommtwity-wide planning of the 
-> f^Voral Jupdred organizations in order to mini- 

''.put* thelUndofrable conflicts, program and personal, which 
often aHse as a result of failure to dear information and sched¬ 
ule* of events. 

The fifth function, personal counseling, calls for some de¬ 
scription As hai been the can on most campuses, many at¬ 
tempts have been made at Minnesota to deal effectively with 
the need for assisting atudent* to choo*e extra-curricular and 
recreational experiences m line with personal needs a* outlined 
m preceding paragraphs, For the most part there attempts 
have taken the form of dispensing information about activity 
opportunities through the medium 3f bulletins, poster*, "Ac- 
txntie* Pay” during Freshman Week, rtudent counselors, and 
faculty ooonidan. Our experience has indicated that tins 
method of counseling through activtty information is probably 
no more effective than is the corresponding method of dispens¬ 
ing occupational information in the field of vocational guidance. 
Some students need only information about opportunities id 
order to choose experiences which prove to be valuable and 
effective. But many other students need both Information 
and personal counseling. Just as the past decade it Minnesota 


has been devoted to the developing of both clinical and faculty 
counseling program* in the field of educational and vocational 
guidance, it ia expected that the present decade will be devoted 
to the development of counseling program* in the areas of stu¬ 
dent finances, discipline, and social adjustment through par¬ 
ticipation in extra-curricular activities Descriptions of the 
evolving programs concerned with financial and disciplinary 
counseling are in process of preparation, 

While the precise dimensions to be taken and the techniques 
and procedures to be used m the projected activity counseling 
are not now known, our experience to date indicates certain 
thing* Jc will not become ARhough it mint bo closely coordi¬ 
nated with mental hygiene counseling provided hy psychiatrists 
and psychologists and with clinical counseling regarding edu¬ 
cational and vocational problems, activity cnnniehng will 
differ from these m that it will extend beyond group therapy 
for serious maladjustments and beyond the measurement of 
aptitudes and interests. It will be as muah concerned with 
‘'normal" students at with those ui Knout emotional difficul¬ 
ties, As now projected, activity counseling will provide assis¬ 
tance to students m the transition from the simpler society of 
home communities and high schools to the more complex 
society of adolescents which we call a university In this sense 
the Activities Bureau wQ] be an agency (but not the only one) 
for the induction of new memben into the college society For 
those students who are able to make their own induction transi¬ 
tion, it will perhaps be unneeded Biit for the large number 
of students who experience difficulties in taking thil step, it 
will offer personalized services to supplement the mass methods 
used at present. Such a icrrico may be unnecessary on many 
campuses but it appears to be needed in this particular type 
of university, It should be emphasized that this typo of activ¬ 
ity counseling is supplementary to many other types of ser¬ 
vice*, some of which are formally organized and others of which 
operate informally It should also be emphasized that par¬ 
ticipation in activities is conceived as only one of several media 
for achieving that degree and type of social adjustment which a 
particular student may desire and need There Is no though* 
of standardizing either the means or the en^s of personal deyri- 
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opment in the are* of social relationships, but rather a type of 
assistance which will aid the atudent in selecting appropriate 
mean* to achieve what he conceives to be desirable and satisfy¬ 
ing personal objective*. Other media for achieving social 
adjustment will bo utilized, including both organized and 
unorganized recreation and personal contact*, attendance it 
ax well aa participation in planning and conducting activ'ues, 
reaadeiitml l.vuig rxpentneef, rone nlir nr.ihipi, and c'sit- 
rooro and perwna] conracii wii'i snidcrta ara tcccben 

While it may apucar in i l c rciH.r of descriptions sheet tin* 
ahd other phases ol M.nie*i>u , i penonnri program that chits 
is a highly compartmentalized organization, such u not the 
case. In a Jarge university, specialization m personnel func¬ 
tions u not only possible but it w also necessary from an admin¬ 
istrative poult of view if the large number of inidsnt* are to be 
served While such specialisation, with its attendant adminis¬ 
trative departmentalization, does introduce difficulties of ad¬ 
ministrative and program coordination, discussed elsewhere,* 
the resulting improvements m effectivenera of personnel ser¬ 
vices is a worth-while gain Incidentally, them specialist* tup- 
■ Jploment but do not supplant faculty counselor*, who serve 
m “general pncutidnera” in counseling 
'.WgW* Activities Bureau, a* is now tnie of other petsonne] 
Apartment*, serves u a focal point of a personnel function 
but not ex an exclusive monopolizer of a function The Coun- 
tdragjBnremi has served u a,pace setter for Faculty counselors, 
»ctu*% b«Ip|bg w increase faculty participation in counseling, 
which tow takes place literally in every building on the campus 
In similar mam rn: ii expeuieii that (J e new Acr./i* es Bureau 
will provide leadership in enlisting cne faculty counselors in the 
UK o' activities ai one mriiit to further pereoiial lirvelupme't 
of siudsnn k n hoped that through du, nrogrim, wilderts a 
organization* and activities will be viewed by more teachers as 
spaying worth-whilo educational and 'professional values rather 
tii«n merely a distracting effect upon students’ Intellectual 
tffbCte. * 
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The beginning of a program of evaluation is being set up 
with the establishment of the Bureau For the first year, in 
addition to those students, who are referred or come voluntarily 
for counseling, a special group of several hundred atudent* from 
the College of Education is being invited to uk the Bureau's 
facilitiu. Thu group vu selected because of the special 
motivation to he derived from the fact that effective profes¬ 
sional adjustment: of teachers is dependent to a large extant 
upon adequate social adjustment. Follow-up studies of these 
students will be made with respect to the detectable outcomes 
of participation Comparisons will be made between this ex¬ 
perimental group and others composed of students who make * 
adjustment* unaided by counselors and thare who fail to pap- 
tic ip ate. An unuMial method of identifying students who may 
need personalized and individualized counseling was developed 
as a phase of the required speech examination of all new stu¬ 
dents conducted by the Speech Clinic. The Qinic’a staff is 
alert to the possibilities in actrvitie* for personal adjustment 
and personality development Such cases are referred t0 the ’ 
Activities Bureau for'personal counseling regarding group-life 
adjustment*. It u expected that the first few yean of experi¬ 
ence will be devoted largely to perfecting counseling jxoce- 
duna, delineating types of adjustment problems, and testing 
criteria rff the outcomes of participation and non-part icipation. 
Among the criteria to be investigated are persistence of par¬ 
ticipation, degree and type of change from high school to col-' 
lege, itibjective reports of satisfaction or dissatisfaction, allevia¬ 
tion of associated adjustment problems, and other factor** in 
relation to leadership or followership and type of activity ,/ 

With the establishment of the Bureau and the initiation, 
an evaluation program, it ie expected that ..the atudent activity 
phase of the University's atudent personnel program will gradu- r 
ally yield experience* Indicative of its true valucp Not the 
least of these values may be the creation of awareness in[ roun- 
■tion of the range of social adjustment problems of student* v 
and also an undemanding of the therapeutic effectivmipf 6£ \ * 
group adjustments os supplementary and coordinate-vrittfiddi^ r “ 
viduiliztd counseling procedures and programs, s ‘ * * 'AA* 

~ ■ ■ \ ' \ / ^ 
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PREDICTION OF SCHOLASTIC SUCCESS IN COL¬ 
LEGES OF LAW II AN INVESTIGATION OF 
PRE-LAW GRADES AND OTHER INDICES 
OF LAW SCHOOL APTITUDE' 


WILLIAM MICHAEL ADAMS 
Uuroii ty d Ian 


Tbis amdy was undertaken with the objective of discover¬ 
ing additional indices of aptitude for successful law school 
achievement which might, in conjunction with tho legal apti¬ 
tude test described in an earlier paper (1), provide a more 
adequate bans for advising students who contemplate entering 1 
the College of Law of the University of Iowa. Specifically, it 
was the purpose of thu investigation to determine the relative 
Validity of pre-law grade-point averages and scores on the low* 
Qualifying ExwtninaUon as predictive indices of success in firat- 
year law at the Univetitty of Iowa. ' 

Because of the variations in grading standards at different 
undergraduate colleges, only those students who had completed 
•11 their undergraduate work at the University of Iowa and 
who had complete records for one year of work in the College 
of Law were included in tho major po rtion of th la study These 
students were selected from entering classes from tho yean 1936 
to 1939, inclusive. The number and percentages selected from 
each dasa are presented in Table 1. 

The predictive indices available for tho group of 152 stu¬ 
dents selected included grade-point averages for pre-law work, 
and scores on the Iowa Qualifying Examination. The Iowa 
Qualifying Examination, which had. been adminatered to all 
die mom bon of this group aa entering freshmen in tho liberal 
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arts college, consisted of the Iowa High School Content Exam i- | 
notion, loaa English. Training Examination, Iowa Math e- | 
mattes A-ptnuda Test, and tho Iowa Silent Reading Test A I 
composite score, consisting of a weighted total of the raw scores I 
an the four examinations, had been computed The raw scores 
on the individual sub-testa of the bacteiy and the composite 
score had been converted to percentiles and were available in 
this lorm It was assumed that these percentiles wore equiva> 
lent from year to year and, for purposes of computation, the 
percentiles wore converted to linear scores by use of the modi¬ 
fication of Hull's table prepared by Guilford (3). 

TABLE I 

A'uei&rr and PntfUi* «f StuJttdj jMiiuy/im Psnow 
Fttiimm CUuu m lA* ColU$» of Law 


Vnr 

Total ■woUcA* 

Total idccnd 

Ponatefa 


ss 

J3 

3852 


96 






3939 

1W9 , 

107 


3925 

Total . 

. 387 

152 

_ 


* Tho ml number enrolled la elch dm included itniltnti who con 
ors'law week at other butiniUoM is wM« or is pm, lime who wnh 
CDtuia of the jru, and Uwm refjtmrad at frc*hnv«n for mom than one 71 


On (he basis of findings of similar studies made m other 
schools of law by Crawford and Gorham (2), Husband (5), 
and Jacobs (6) ( it vu decided that it might he profitable to 
make separate studies of the predictive value of the total pre¬ 
law grade-point averages, the junior yoar pre-law grade-point 
averages, the pre-law grade-point averages in social sciences, 
and the pre-law grade-point averages w physical sciences, as 
well as of the scores on the Iowa Qualifying Examination (or 
this group 

The available criterion of success in firet-year law consisted 
of a weighted total of the numerical grades assigned in each law 
course completed by the student during his freshman year in 
the college of law These weighted total grades were convened 
to ranks for each class, and the ranks were converted to linear 
scores by the method suggested by Hull (4) These linear 
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scores commuted the actual criterion measures utilized in the 
correlational analysis 

The performance of the criterion group of 152 students in 
the aptitude tests and pre-law areas studied is summarized in 
Table 2. 

TABLE 2 


gitUMi Pttftn 


Pwlam Worn 


fow. Omlikmi Emminttkr*. 

1 Cmpo.It* un . . 

I!2 

3197 

if. n 

Lbttr tears 18-9J 
PttcnDlc W9 

Linear (ten 2t-W 
PtewstSs 7-09 
Lwcat »ew« 7-OS 
PsrwittSs LW 

hSJrtT* 'SS 

2, R((l SaAooICtMnl . 

1S2 

HYI 

14.17 

3. IfMAsstelki AsipaAr . 

m 

MUX 

18X3 

4, JTafBth JrmMai ,, 

1S2 

5351 


1 Sbu KtUImg . . .. 

in 

nm 

16J0 

Li art r ton 11-W 
PuwutJt 2-99 

Tool ft»Jaw dr A., . 

Phj*. n. mvIsw G ? A. 
T«msn-bvaf A .. . . 

m 

in 

153 

in 

III 
U7 

if, 

JOt 

553 

M 

ft 

US-394 

147-4 ID 

1SU 

1J2-394 

Jr w pir-Nwaj A. 

T«til f»lt« UFA, , 

361* 

ut 


D 

157-4X0 

IM-iJS 


* A npplBUMirr inity »u mli «f 361 nultnn 
T A wesu ntpbMDtur rntf wu mads of the rw of 81 iraritcou 

The mean linear koks of this group on the Iona Qualifying 
Examination and its sub-tests Were definitely superior to the 
scores of liberal arts freshmen as a whole, hut the range wu 
very wide, varying from scores corresponding to the fifth to the 

ninety-ninth percentile in the composite iKore- Thq mean total 
pre-law grade-point average* of 2JB for this group wu also 
Superior to the gride-point average of approximately 220 
achieved by students of the College of Liberal Am u a whole, 
but probably not much superior to that of liberal arts seniors. 
Again the range U wide, extending from a grade-point avenge 
of 1 55 in rare of 3 94 From there dir* *r would appear thar 
the typical University of Tow* indent who enters the College 
cf Lawn Iowa u dr finite’> »upeno r , ■■ a freshman, cii the few* 

, * *• u,h ” 10, to " * 
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Qualifying Examination and maintains a somewhat better than 
average record in hw liberal arts work. 

Inspection of Table 3 reveals that a moderate correlation of 
.47 wu found between the composite scores of the Iowa Quali¬ 
fying Examination and the criterion of success in first-year law 
for die 152 students who had received all then pre-law training 
at the University of Iowa A multiple correlation of only iQ 
was obtained between the criterion end an optimum combina¬ 
tion of the scores on the four sub-tests of the Qudlifying Exami¬ 
nation lor this group. Of the indices studied for this criterion 


, TABLES 

Camtatm el (A* Pr4chw InAwi trilA iAj Cnterfo* 


lopm OtuUfynit Emmie* Hen 
.Mima ii(i 


al prHis mirk 

SL ‘ ‘ 

npnl irlisff , 
T«i|p»k«wt . 
JuMw jbi .... 
Tool preJw aerk 


ft 

JH 

.451 


i Widen a who bid tab** 


group, the total pro-law grade-point overage appears to camj 
suture the moat valid single index of aucceu m first-year l*p$ 
correlating 67 with die critenon. A multiple correlauon sQ 
only f® was obtained between the criterion and an opumUdr| 
combination of total pre-law grade-point averages and con^f 
posvte scores on the Qualifying Examination. Apparently adfjf 
ditto n of this latter wide* adds little to the predictive value 
total pre-law grades. 

Since it was found that, over the period of the study, 
dents entering tha University of Iowa College of Law had < 
pie ted all or part of their undergraduate training at 85 diffei 
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institution*, supplementary studies were made of available pre¬ 
dictive indices for two different groups The lint of these 
Studies, baaed on the undergraduate transcripts of 361 students, 
regardless of the institution in which they had taken their pre¬ 
law work, yielded a correlation or 56 between the criterion of 
success in lint-year law and total pre-law grade-point averages 
and a correlation of 55 between the same criterion and junior 
year pre-law grade-point averages. A second supplementary 
study, based on the undergraduate transcripts of 83 students 
who had completed all their pre-law work at institution* other 
than the Univemty of Iowa, yielded a correlation of only .45 
between total pre-law grade-point averages and the criterion 
of first-year law school performance. It would appear, there¬ 
fore, that total undergraduate grade-pouit average! do not con¬ 
stitute a very effective index of scholastic success ut law for 
students who enter the College of Law of the University of Iowa 
from other undergraduate institutions. 

lit onlor to evaluate the possible efficacy of requiring a 
Bachelor’s degree as one of the criteria for admission to the 
College of Law of the University of Iowa, the first-year law 
grades of 212 students who had entered the lew school pnor 
to obtaining their Bachelor’s degree* were compared with the 
corresponding grades for 149 students who had received their 
degrees before entering the law school. A critical ratio of 2 13, 
significant sc approximately the 2 per cent level of confidence, 
wa* found between the means of the linear criterion scores for 
the two groups, indicating chat the former group was actually 
superior in first-year law school performance to the group who 
had received their Bachelor’s degrees before entering the Jaw 
school. Thu finding u in agreement with the results of similar 
studies ui several other law schools. A further study of these 
two groups revealed that the former group was slightly, al¬ 
though not significantly, superior to tha latter group with re¬ 
spect to total pre-law grade-point averages. Apparently selec¬ 
tive non-imellective factor* such at motivation, chronological 
age. and continuity of college attendance, as well as scholastic 
aptitude, must be Invoked to account for the supenority of the 
student* who entered Uw school after only three yean of pre¬ 
law preparation 
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On the basts of the findings of the studies presented in this 
and a preceding article (1),> the following conclusions seem 
justified; 

1. The experimental edition of the lotaa Legal Aftitmda 
Test appears to show considerable promise as a predictive index 
of scholastic success in first-year law for populations similar to 
those included in this preliminary investigation, as evidenced 
by correlation coefficients of from .48 through 76 hatween total 
score and first-year law achievement in five colleges Further 
experimentation with this test, involving larger criterion groups 
and utilizing more ngoroul statistical techniques, should even¬ 
tually result in a much shorter and more valid final edition. 

2 Of the predictive indices, other chan the legal aptitude 
test, studied in the present investigation, total pie-law grade- 
point averages appear to constitute the best single Index of 
success in first-year law for students entering the College of 
Law of the University of Iowa from the undergraduate division 
of this institution The validity of this index is considerably 
higher for students who have taken all their pre-law training 
at the University of Iowa than for students who have taken 
part of their undergraduate training elsewhere. For students 
who have taken *11 their pre-law work at ether undergraduate 
institutions, tho predictive value of pre-law grades alone is too 
low to be of much value for individual prediction 

3 Composite and sub-test scores on the lotaa Qualifying 
Examination do not seem to be sufficiently valid indices of 
scholastic success m first-year law to be very useful for pur¬ 
poses of individual prediction at the University of Iowa. Com¬ 
bination of these indices with total pre-law grade-point aver¬ 
ages by the method of multiple correlation does not materially 
increase the predictive efficiency of this latter index 

4. In a comparative study of the relative scholastic success 
in law of students who entered the College of Law of the Uni¬ 
versity of Iowa after only three year* of undergraduate prepa¬ 
ration and students who received their Bachelor’s degree* prior 
to entrance, no evidence wai found to indicate the desirability 
of requiring college graduation aa a prerequisite for admission 
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to tho college of law insofar as acholasttc achievement in first- 
year law was concerned However, it should be remembered 
that the requirement of a college degree might be fully justified 
on other bases not investigated in this study 

5 A properly weighted combination of scores on the lotaa 
Legal Aftottide Test and total pre-law grade-point averages 
appears to constitute the heat quantitative index of success in 
firit-year law at the University of Iowa available at present, 
as evidenced by a multiple correlation of .77. It is possible 
that the legal aptitude test scores alone will yield the bat pre¬ 
diction for students who have taken moat of all of their pre-law 
training at institutions other than the Univemty of lows. 

6. Improvement of our ability to predict seholaatic succesa 
in law will probably result From experimentation with optimal 
combinations of pre-law grade-point averages, scores oil im¬ 
proved tats of legal aptitude and, possibly, comprehensive col¬ 
lege achievement examinations and objective measures of non* 
intellective characteristics as yet not adequately subjected to 
quantification Of three latter, such factors as interests, per¬ 
sonality, perseverance, health, and personal history data in gen¬ 
eral in in need of further investigation Further analysis of the 
intellectual functions required in legal studies and the develop¬ 
ment of more specific tests for their measurement are needed. 
Such studies will be facilitated by all improvements in the relia¬ 
bility of the criterion of success itself 
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A TECHNIQUE FOR EVALUATING ASSEMBLED 
EVIDENCE OF POTENTIAL LEADER¬ 
SHIP ABILITY 

a x. MANBON XHB O L. FREEMAN 

lfartkwatara Unbonby 

Tm rapid expansion of our armed foreei and the growth 
of new government agencies has created a shortage of men of 
proved executive talent. Consequently, the personnel expert 
is under considerable pressure to develop methods for selecting 
men of foiential leadership ability In this task, ic is essential 
to pool such pertinent information ss is available. The pres¬ 
ent report dealt with methods used In selecting a group of 
young men with aptitudes for fausinesa leadership It is 
offered ss one approach to a difficult problem of personnel 
selection 

A number of large companies with a policy of training col- 
lege graduate* for executive positions have been faced with tho 
problem of appropriate selection. Such companies rarely re¬ 
veal their selection techniques, and consequently there i* little 
available information upon the subject. This was the situa¬ 
tion presented to Northwestern University's Personnel De¬ 
partment fifteen yean ago when it hid to select from high- 
school seniors throughout tho country, ten young men who 
would be subsidised in a special training course for ^executive 
work in the field of business “ The group, known as The 
Austin Scholars, were to receive a stipend covering the expenses 
of four yean of college plus a year’s travel abroad As a result 
of this subsidy there was no dearth of applicants, and standard¬ 
ized procedures had to be instituted Tor their progressive elimi¬ 
nation. 

The method of selecting Austin Scholars evolved around 
four interlocking problems These were (1) making a jpb 


l 
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analyau o( tKo duties an<Tqualtuta of business (cadets, (2) 
securing a population sample likely to contain potential busi¬ 
ness executives and assembling information relevant to its lead¬ 
ership qualities, (3) judging upon the baais of assembled evi¬ 
dence the relative degree to which individuals possessed the 
desired qualities and (4) selection from the top rank* of rater 
judgment upon the basis of personal interview Each of these 
problems will be discussed ui ium- 

I. Job Analytu 

In order to make an intelligent selection of men to be 
trained (or future executive positions, it was first necessary to 
ascertain the special qualities found in persons who were already 
successful on the job Under the auspices of its director, Dr. 
D T. Howard, the Personnel Department of Northwestern 
University prepared suggestions for a functional job analysis 
and submitted these to recognized business leaders for correc¬ 
tion* and recommendation*. The analysis which emerged 
paid attention both to the factors held m common by execu¬ 
tive* and all men of talent and to the dtgarentia of the job,— 
that is, the functions, methods, and abilities which distinguished 
it from other types of work Among the factors common to all 
men or talent primary weight was given to intelligence. The 
differentiating features were less easily defined through study 

of the function* of the bushes* executive* These were Ii*tcd 
as follows i 

1 The executive must secure and evaluate information 

2. Ho is required to be a planner and work-org.uuzor. 

3. He must engage in conferences and communicate ideas 
effectively 

4. He must assume responsibility, making decisions and 
standing by them 

5. He mint maintain effective interpersonal contacts and 
have ^ reputation for pereonal integrity with both his 
employees and associates. 

The personal qualities judged most appropriate to the suc¬ 
cessful execution of the above functions were divided into five 
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classes: (|) physical tram and endowments likely to favor 
effective communication of idea*, (2) mental abilities essential 
to the planning and organization of work, (3) social talents 
involved in the cooperative and directive phases of executive 
performance, (4) character traits contributing to confidence m 
one 1 * integrity, and (5) cultural accomplishment* making for 
pereonal distinction, leadership, and prestige. 

II. Sampling Procedure 

With the foregoing analyte* as a guide, the Personnel De¬ 
partment next proceeded to secure its sample of high-school 
■enton and to obtain information of likely value in estimating 
the leadership potential of the individuals under consideration. 
On the assumption that high-level Intelligence it a necemry 
starting point for further screening, this wo* the lint point of 
emphasis Letters were sent to over a thaiuand high-school 
principal*, asking them to post a circular describing the Austin 
Scholarship Program and to recommend their roost promising 
student* who met the following qualifications: 

Only thaw students will be conildared who stai 
tenth w their graduating classes m scholastic achre 
conn must be of sound health, able to talk well and 
be loaders among their fellows, and must be respected in thar rom- 
mututio* for thar moral and personal qualities. They must not have 
previously attended college elsewhere, and no person need apply nn- 
«*■« “ his intention to prepare himself for executive work in the 
field of busmen 

In the year covered by this report 441 young men signified 
their interest by inquiring about their eligibility for Austin 
Scholarships Of this number 109 did not follow their inquiries 
with formal application and an equal number wore found In¬ 
eligible for reasons ranging from lateness of application to lim¬ 
ited scholastic requirement*. Thia left 223 young men who 
satisfied all technical requirement* They came from 35 dif¬ 
ferent state* and the District of Columbia. A folio compiled 
for each of these applicants earned the following information' 

Form A. Personal history, including photograph, place of 
residence, schools attended. Interests and active 
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ties, oatnde employment, and future occupa¬ 
tion*! plana , 

Form B, Complete record of health from family physi- 


third of the composite "order of menc** list. Thu procedure 
of progressive elimination firit reduced the list or 223 applicants 
to BO applicant* and finally to 25 applicant* It should also be 


Form C Complete echolaitlc record from high-school 
principal 

Form D Rating* on five personality attributes (leader¬ 
ship, originality, popularity, speaking ability, 
and character) secured from three person* well 
acquainted with the applicant. The complete 
rating form, referred to a* Form D, is given 
herein 

Form E, Standardized intelligence teat {Psychological 
Corporation, Tati VI) given by high-school 
principal and returned ro the Personnel Bureau 
for scoring. 

Form F. Standardized English placement teat {.Columbia 
R»storch Bureau English Test) given by high- 
school principal and returned to the Personnel 
f Bureau for scoring. 

Form G, Correspondence and supplementary letters 

III. Rating Assembled Evidence 

With 223 superior applicant* technically eligible for the 10 
Aujtip Scholarship*, some process of futther screening had to 
be set Up so that due consideration could be given each cim in 
the assignment of relative rank. The technique finally devel¬ 
oped involved five steps. (1) breakdown of list of candidate* 
into group* of thirty each, in order of completion of applica¬ 
tion; C2) further breakdown within each group of thirty, ao 
that three rater* independently ranked the leadership qualities 
of each candidate relative to a group of ten applicants, (3) 
rejection of twiwhlnl* (approximately) of the total group, 
largely upon evidence from objective texts; (4) teranking of 
remaining candidate* (in groups of ten) by three new rateri, 
and (5) combining, for each candidate, the rankings received 
from the*U rater* and then selecting for further study by inter¬ 
view those candidates who composed approximately the upper 
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FORM D—(CwuluMfl 
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•TWtlbn, wa pnpand hy Dr D T, Howard, 
noted that as a result of the methods used, each candidate in 
the fine elimination series received a man-to-man companion 
with three different groups of ten candidates each, and that 
those who survived to the second elimination senes received 
similar man-to-man companions with three more groups of 
ton candidate* each. A random sampling technique was used 
for placing a candidate in the ten-unit groups assigned to ntorv. 
While there was some duplication, the total constitution of 
each group was thus different for each rater By thia method 
each applicant was assured of a ranking by each of three men 
(6 for those who survived first elimination) and waa compared 
with a number of other candidates, but not in groups ao large 
m to make relative ranking difficult or meaningless The de¬ 
vice tended to prevent inequalities which might appear if the 


different groupa of ten each wen, as groups, unequal in ability 
or achievement. It also made for efficient work-organization 
in the Petionnel Department, since it enabled raters to start 
on the elimination problem before all applications were com¬ 
pleted and at timea which fitted into their ocher work-sched¬ 
ules That the rating method waa fairly reliable is indicated 
by the following mtereorrelationa between raters In the first 
ranking procedures, reliability coefficients between judges were 
as fallows* 1 and 2, 70, 1 and 3, r=+^9, 2 and 3, 

73 In the second ranking procedure, reliability coeffi¬ 
cients between judges were as follows: 4 and 5, rs=+ .54; 4 
and 6, r = + .61; 5 and 6, r ■= + .73 When rater evaluations 
given candidates who survived both competitions wen com¬ 
pared, the following mtereorrelationa resulted i 
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The size of the individual reliability coefficients indicates 
that there were relatively few wide variations in the rating of 
applicants. Examination of the cable shows the usual ten¬ 
dency for reliability to increase as the final score become* w 
average of a greater number of ratings A reliability coefficient 
of .91 would indicate that the technique of rating is about as 
consistent as the usual objective examination 

We shall next discuss the qualifications on which the candi¬ 
dates were compared in the above procedure. There were 14 
of these qualifications and these were considered one at a time. 
The procedure was to examine the assembled information con¬ 
tained in the folder of each of the ten members of a given 
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group with respect to the quality under consideration. The 
individual judged to have the moat favorable manifestation of 
the quality was arbitrarily given the highest rank of 10, and 
the applicant with the least favorable manifestation a rank 
of 1 The final ranking was the summation of the individual’s 
positions on the 14 qualities bsted below, after the leadenhip 
rank had been given a triple weighting and the character score 
a double weighting. 
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It will be recalled that two further items of information 
hive not been used in the above evaluation of applicants. 
These are the scares on standardized teats of mental alertness 
(Form E) and English (Form F) Although our present 
approach would be to combine such objective test rankings 
with die total rating score, in 1932 the objective tests were 
u *“ *J* eliminate two-thirds of the applicants who composed 
the original sample. That is, those applicants who failed to 
plate within the fifth decile 1 or better on both testa were ex¬ 
cluded from further consideration. Such a drastic procedure 
was dict ated largely by a desire on the part of the University 

lta l "SrSJT e &*' T «w*Wd.wn. U7+r«t*E ^ 
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that, above all, the men selected should excel in scholarship* 
V/c now quel lion i( such an oveiweighting of the intellectual 
factor waa necessary in view of the high academic record of the 
applicants in high achool 

Reviewing the steps taken in selecting Austin Scholars, we 
have* 

(1) Rating all qualified applicants on fourteen qualities by 
three (Group A) raters. 

(2) Elimination of applicants who did not have a fifth 
decile or better standing on both the psychological and 
English examination 

(3) Re-rating the 14 qualities of applicants who have met 
the standards (imposed in 2) by three new (Group B) 
raters. 

(4) Eliminating the lower two-thirds of applicants on the 
Group B ratings (imposed m3), and calling the upper 
third (here 25 applicants) for personal interview 

IV The Personal Interview 

It may be recalled that up to this point, the Personnel De¬ 
partment had only indirect contact with applicants and had 
made its evaluations upon the basis of assembled evidence. It 
was deemed advisable, therefore, to base the final elimination 
upon a personal interview These interviews were bandied by 
the Director of the Personnel Department, with some assistance 
from business leaden in the ca>e of certain candidates The 
person conducting the interview did not know the previous 
rank order of the applicants referred to him. Consequently, 
he could obtain unbiased impressions The thing most sought 
In the appraisal interview was evidence of mental agility, earn¬ 
estness of purpose and ability to hurdle oneself effectively in 
a difficult situation. The results of the interview were now 
evaluated against the evidence already assembled on the 25 
candidates, and the final selection of 10 was discussed in con¬ 
ferences between the Director of Personnel and the six raters 
who had participated in evaluating evidence. In terms of their 
relative ranking on assembled evidence (Part III), the ten 
selected stood 1st, 4th, 5th, 8th, 9eh, 13th, 14th, 16th, 20th, and 
lass in ih« group of twenty-five. 
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From a fifty per cent return of questionnaires sent to scholar 
and .non-scholar groups five years after their graduation from 
the University, it appeare that the five Auttm Scholars who 
replied earn higher salaries, do more supervisory work and 
receive more promotions than do the five n cm-acholaT* who 
replied. Avenges are given in Table 2, together with Fisher's 
t tests of the significance of tho differences found. 


TABLE A 
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VI Comment 

It is recognized that the proposed technique (or selection 
of potential leaden has received no precise validation. Never¬ 
theless, Its internal consistency might commend it to business 
organizations, mititaiy boards, and government agencies who 
have to select from a large number of applications for a given» 
job. By using a rating form appropriate to the job, the sorting 
of application papers can proceed in an orderly fashion. The 
authors recommend especially that applications be assembled 
m groups of thirty as they are received and that this number be 
broken into three random groupings of 10 each, so that each 
application ip evaluated by three different raters in reference to 
a somewhat different sampling. By considering only tho top 
third ranking* (avenge of dime rater*) from each group pf 
thirty, a large number of applications can ihortly be reduced 
to snore Workable proportions for a second comparison by a 
fresh group of equally competent raters If necessity, the top 

thlrdeaa again be skimmed, from the second comparison and 
.arranged (far re-ranking a third time. By cuch a method, the 


V Evaluation of Method 

The personnel worker re naturally interested in the outcome 
of the men selected by our method. Unfortunately, it has not 
been found possible to complete fully the file of each Austin 
Scholar five years after graduation But the fact that a num¬ 
ber of Austin “rejects" an rolled in Northwestern University of 
their own accord makes possible certain group comparisons 
between scholars and non-scholars The grade-point averages 
for 10 Austin Scholars was 5 73 and for 20 non-scholare war 
4,90 This difference approaches significance (GR. 21), 
Furthermore, a numerical check of campus activities during 
the four years of college gave the Austin group an average 
count of 12 and the non-Austin group an average count of 7 
activities The difference is not stans tically significant. 

Such evidence of leadership ability is difficult to evaluate 
and a better test is a comparison of the scholar and non-scholar 
groups in terms of achievement after graduation. We selected 
for this purpose ten men from both the scholar and the non- 
scholar groups whose post-school records were complete In 
the non-scholar group we find, soma time after graduation, a 
salesman for a meat-packing firm, a research associate for a™ 
peace organization, a minor civil service employee, an auto 
salesman working for hia father, a salesman for a hook concern, 
an accountant, a chemical engineer, a high-school teacher, an 
office assistant in a Urge grocery chain, and the assistant gen¬ 
eral manager of a national manufacturing organization. The 
scholar group contains, besides accountant* for two large In¬ 
dustrial organizations, two specialists in industrial engineering, 
a high-school teacher and five men who have risen to mana¬ 
gerial authority in Urge corporations, Included in this most 
successful group of former Austin Scholure is the associate edi¬ 
tor of a national magazine,, a purchasing agent, an executive 
assistant, a plant superintendent, and an assistant sales man¬ 
ager. While quantification of such evidence is difficult, it 
would appear that many Austin Scholars have done well in the ' 
busmen world even during an economically unsettled decade. 
The above evidence la influenced by the training given to 
Austin Scholars as well as by the selection method 
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men finally chosen will have been ranked by a number of raters 
relative to several groups of applicants and in reference to a 
fixed sot of criteria 

The author* hold no special claim for the specific criteria of 
leadership ability used ui the present selection procedure. 
These aro probably no better nor worse chan the Usd of quali¬ 
ties now in use elsewhere in the rating of leadership potential 
It should be pointed out that before any such list of qualities 
can he used with precis ion, it will be necessary to determine the 
degree to which each item discriminates between leaden of 
demonstrated ability and non-leaders. Until then, the relative 
weighting of Items on a leadership rating scale must rest largely 
upon the opinion of the examiner The authors feel that ab¬ 
stract intelligence was overweighted jn the selection of Austin 
Scholars, The feet that this type of overweighting is charac¬ 
teristic oi most leadership selection procedures is no justifica¬ 
tion foe its continuance. It teems that penoimri worker* 
might well give further study to an alternative proposition. 
This proposition is that since leadership u a social quality, it is 
better fudged by evidence of effective social Interaction than 
fay non-social testa of individual ability. 
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JUDGMENTS IN COUNSELING 1 


RALPH F BERDIE 
Um«nlij of MIabmob 


Vocational countelora in student personnel programs at¬ 
tempt to assist students make judgments about their awn 
capacities, potentialities, and personalities The counselor 
supplies the student with two things Primarily, he gives the 
student information about himself and his relationship to his 
surroundings The student u told haw his various abQities 
and interests, as estimated by test scores and past achieve¬ 
ment, compare with the abilities and interests of people In dif¬ 
ferent jobs, schools, and curricula. The counselor then helps 
in the use of this information so that the student can make a 
sound judgment regarding plans for the future. 

Techniques for estimating abilities and interests have been 
discussed by many writers and summarized by others (1) 
Methods of assisting students m arriving,at judgments have 
been discussed by YfiHiamion, Rogen, and other* (4,2) Be¬ 
fore a student can Understand his situation and plan for tho 
future, he must make companions and discriminations In¬ 
formation that has been analyzed and evaluated results In 
judgments, which in turn lead to decisions, which in their turn 
lead to action. 

An attempt to adopt merhods of clinical psychology and 
psychiatry in vocational counseling has led to n difference of 
opinion regarding underlying philosophies and techniques. 
One group of workers emphasizes the judgments that the coun¬ 
selor must make. Another group expresses the opinion that 
it is not only unnecessaiy but alio undesirable that the coun¬ 
selor make judgments. The counselor, according to the latter 
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group, functions only to assist the student in amving at judg¬ 
ments and does not provide judgments of his own which the 
student can accept or reject. 

The view of the group deeming the counselor's judgment 
essential is expressed hy Williamson (4, page 118)* 

It should be repeated (hat m vocations! counseling the counselor 
collects data shout the student'e potentialities and than proceeds to 
compare these data with the requirements of the student's expressed 
occupational choiee In this way he smves at a diagnosis of apti¬ 
tude and ■ judgment as to the wisdom of that choice. This com¬ 
panion of poUitiMklMJ vhJt preferences is the important- Jtsp in 
diagnonrJ Following this step comes the interpret*don to the stu¬ 
dent of the cDunulori diagnosis and the cooperative planning of 
next steps. 

Hie judgments studied in this investigation were judgments 
made by counselors about students, and the results are there¬ 
fore most relevant for those counselors subscribing to the school 
represented by the quotation from Williamson 

When a student seeks vocational counseling, he usually is 
considering one of more possible vocations and he is attempting 
to make a judgment regarding the relative appropriateness of 
the considered vocations before deciding to enter one. The 
large number of factors serving to attract a student to various 
vocations results m one vocation usually being more or less pre¬ 
ferred over others. Students seldom are considering two or 
more vocations all of which seem of exactly equal attractiveness 
They have preferred vocations and they want to determine if 
they should enter those vocations To assist in this determina¬ 
tion the counselor and the student organize information based 
upon school grades, aptitude and ability teats, achievement 
tests, interest tests, personality scales, hobbies, and work ex¬ 
perience. This organization usually results in both the coun¬ 
selor's and the student’s making judgments about the appro¬ 
priate nesa of vocational choices The counselor's judgments 
may be the most effective force in molding the student’s 
judgments. 

This study sought to determine how consistent different 
counselors were in making judgments about vocational choices 
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and what factors were instrumental in their arriving at these 
judgments. The judgments were all baaed upon the tame 
material, it, case folders, and the counselors all had gone 
through a senes of training clinics that supposedly supplied 
them with a common knowledge of the relevancy of the test* 
used and the data collected 


Methods 


Twenty cue folders of pre-college men were selected from 
the file* of the Testing Bureau of the University of Minnesota. 
The vocational choice of each of there students had been judged 
m the first counseling interview as either appropriate or inap- 
propnate by the counselor. The case notes were then reviewed 
by a care reader who verified the judgment of the onguial coun¬ 
selor A duplicate folder was then prepared for each of the 
cares and all identifying data, such as name of student, name 
of father, etc, were removed Hie Following material wu con- 
tamed in the folder in this order; 


1 


4. 

5 

6, 

7 . 


The preliminary interview report—containing the student's 
nated problem, hie high-school record, and observations made 
by the receptionist 
The r 


- -B summary profile of rest scores—containing the names of 
the tests, the raw scares, the percentile scores, end the norm 
groups. Ten* given ac the Tearing Bureau include teats of 
scholastic aptitude, acholasuc achievement, special abilities, 
and personality 

The Strong Vc&tioiml Interest Blank profile. Sprea were 
recorded for twenty-eight occupational scales and two non- 
oecu^aaoazl scales, masculuucy-fcmluuucy, and occupational 

The Kudcr Preference Record profile, un revised form These 
profiles were not available for all of the twenty cases 
The University Tea nog Bureau Individual Record Foret —a 
qnretHMiujre covering family,background, work experience, 
rearstional activities, occupational plans, health history, eic. 
A short statement of factual information obtained by the 
original counselor In the fust interview with the student. No 
interpretative statements v 


A racing form_ _ _ 

The icudent cam** to you and says, “If*you say 'yes,' I 
will attempt to become a(n) (tiuett occupational choice ) If 
you say 'no, I will not" 

In this are you are forced to mike » judgment. On the 
bans of the contents of this folder alone, would you say: 

Yes.. No. . (check one) 


the three following statements, 
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L II you think someone might disagree with you on the above 
J iSdb*^ what do you think his bans lor disagreement 

2. Why do you dunk this basts Insufficient to change your 

judgment? 

The folders were then arranged in numoncnl order and 
gtren to each of the five counselors wjtfi the following 'instruc¬ 
tions 

This is a necessary step in determining how judgments regarding 
the appropriateness of students' vocational choices are made apd 
how comment these judgmenra are. Will you please read through 
the twenty folders in the order in which they ore given to you and 
after you have obtained tho necessary information is you do in the 
mcsrviaw or prior to the interview, answer tho questions which are on 
the mimeographed sheet* in cadi folder. 

It is impossible to duplicate the exact Interview situation without 
the student, but in order to provide is much of the interview atmos¬ 
phere as posable, the data obaerved and recorded hy the ongmal 
counselor have been included in the folder. Regard these obierva- 
t»Dt as factual rather than interpretive, is, they are exact stated 
mem* of description rather than what the observer thought Please 
be sure to answer all the questions 

After each counselor had rated the twenty cases, a pro¬ 
cedure requiring From two to four hours, the folders were 
checked and put in their original order New rating forms 
were inserted and the foldora were given to the next counselor. 

, Results 

Table 1 shows those cases judged by cadi counselor a* hav¬ 
ing appropriate vocational choices and those having inappro¬ 
priate choices The check after the number of a student indi¬ 
cates that the counselor judged his choice as appropriate. The 
absence of a check indicates that the counselor judged it as* 
inappropriate. 

Of the 100 judgments, fifty-five were in the direction of ap¬ 
propriateness and forty-five m the direction of inapprapriate- 
ness. Of the twenty students, ten had been Judged by the 
original counselor as having appropriate vocational choices and 
ten a* having inappropriate choices. Eighty-one of the J00 
judgments made by the five counselors agreed with the judg¬ 
ment previously made by the counselor who originally inter- 
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viewed the student. Analysis of the data m Table l yields a 
tdfrachonc coefficient of .86 between the judgments made by- 
the original counselor and the judgments later made by the 
five counselors. TTie results indicate that in judging the ap¬ 
propriateness of the vocational choices of groups of students, 
trained counselors are able to agree, 

TABLE 1 


Jadrewau °l CoMVilan Ct***mn Af*n>fnolt**u of i*a FoMtW 
CkausM ai Tmtull SIiUmmU 



Of the decisiana originally made in the direction of appro¬ 
priate choice, 14 per cent were later judged as inappropriate. 
OF the decisions originally made in the direction of inappro¬ 
priate choice, 24 per cent were later j'udged as appropnate. 
This suggests that counselors tend to agree more upon judg¬ 
ments when students’ choices are appropnate than whoa they 
are inappropriate. Counselors are perhaps more Willing to put 
their stamp of approval on the student’s plahs than they are to 


discourage him. If a doubt exists in the mind of a counselor I 
regarding the appropriateness of a student's vocational choice, | 
he fa more apt to decide the choice warrants a try-out than to ! 
decide it does not These generalizations spring from clinics] * t 
experience and the above data tend to substantiate them. J 

The judgments of the majority of counselors (three M ' 
more) agreed with the original cla ss ifica tion of appropriateness- ’ 
in appropriateness in eighteen of the twenty cases. The only j 
people who considered ehe choice of case 14642 appropriate ‘ 
were the original counselor in both his first judgment and later ' 
judgment, and the case reviewer. One counselor besides the ' 
original counselor and the case reviewer considered the choke 
of case 13964 inappropriate, but three counselors considered it 
appropriate. The results do indicate that in the great major¬ 
ity of cases, judgments of the five counselor* agreed with the 
decisions reached by the original counselor and the caw . 
reviewer 

Fourteen of the twenty case* had originally been inter- 
viewed by counselors who later served as judges in this study. 

It u doubtful if the original counselors remembered the ,■ | 
u all identifying data were removed at second presentation 
and a period of several months had elapsed since the original 
interview Of these fourteen judgments, not one disagreed 
with the anginal decision. Even those two cases on whom the , 
majority of counselors disagreed with the original counselor 
were consistently judged a second time by the original coun¬ 
selor. Here is evidence that a period extending from two to 
four months does not alter the judgments a counselor makes 
regarding the appropriateness of vocational choices and that 
counselors tend to arrive at tho same judgment in the inter¬ 
view as they do by merely reviewing the case data and never 
seeing the student. These are essentially the results found by 
Super and Brophy (3), and it seems safe to generalize from 
them m the area of vocational guidance. 

Up to this point, the analysis of judgments concerning ap¬ 
propriateness and inappropnstenesi of vocational choices has 
provided Information regarding the consistency of these judg¬ 
ments, Little has been shown regarding the actual judgment- 
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making processes themselves, however, and* the facto is con¬ 
sidered in these judgments have not been discussed, A more 
complete understanding of these judgments will be obtained 

through a careful consideration of some of these cases them¬ 
selves and thd comments made by the counselors as they were 
making the judgmenu Limitation* of space prevent presen¬ 
tation of this material for all of the twenty cares 

Car t number J3755 was a seven teen-year-old hoy who grad¬ 
uated from high school one month after be carat to the Tear¬ 
ing Bureau. Hw occupational choice was chemical engineer¬ 
ing, hu father was dead, ho had one eleven-year-old brother, 
and ho said he would have to work about twenty-four bourn 
a week while attending college 
Teat scores were as follows; 
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Strong Tut scores were A on keys fop eheptut, farmer, 
carpenter printer, mathematics and physiol science teacher, 
forest auviee man; B+ on keys for dentist, engineer, Y.M.CA. 
physical director, social science high-school teacher, office man; 
B on keyi (or production manager and accountant. A ftuest- 
knity^Fetnininlty percentile—82; Occupational Level jjercontfle 
—2 Percentile icons dn the Kuder Preference Record were: 
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Scientific, 98, Computational, 30, Musical, 8 ; Artistic, 86; 
Literary, 8; Social Samce, 9, Persuasive, 44. 

The following statement of factual information was ob¬ 
tained from data contained in the interview notes of tho coun¬ 
selor who originally counseled the student. 

He is a husky, rather ordinary appearing young man He is a 
Uttk crude in his approach but very friendly His lather to dead end 
ha lives with his mother and younger brother 

Ha will be totally •clf-supporung. He has saved about 9100 from 
hu paper route ui the last rnree or four yeen and pill continue to 
have a morning and evening route while going to allege 

He nye he cook his lint tiro yean m luah school ee a joke end as 
compulsory education, Then he realized he must work asrder but 
was unable to fully compensate for hla prtjvwua loafing. He actum 
well motivated. 

The original counselor and the ease reviewer onginelly 
decided that the buy*! choice of chemical engineering was ap¬ 
propriate. In making Hie later judgments, four of die five 
counselors agreed that this choice waa appropnate; one decided 
it waa not. 

The dissenting counselor's response to the first question 
concerning the basis for disagreement was ae follows j 

They would stress the adequate ability and high-echool achieve- 
nunc factor*, together with the A and Bf scores on the chemist end 
engineering key on the interest rest. They would then fullest that 
the boy bp oven a chance to show what he could do They would 
also stress the good tepta of background in nuthemabea and chew 

in response to the second question, concerning bis reasons \ 
for considering the above inadequate to change his judgment, ■ 
be wrote- s 

The discrepancy between H.S.H. and A C\£. or Ohio seems to<. 
Indicate un Jet- achievement of lone irantfing. The narrowly 
Occupational Level score and die problem or complete financial sup-- 
port ere not hopeful signs of staving power The block of iktlW 
trades interests on the Strong Tejl h vitally important, partially 1 
because of their rarity, Jj 

Three of the other counselors mentioned the low high-school $ 
rank but did not appear to give it enough weight to have thortJ 
change their judgments One counselor thought a basis fof^ 
disagreement might be found in the (act that it was too esrfj^t 
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for the fray to make a choice as Bpecific as chemical engineering. 
Two counselors specifically mention the boy's motivation and 
appear to think that the fact that his achievement testa arc 
better than his high-school rank improves his chances in the 
umvcraity, 

One counselor, rating the occupational choice as appro¬ 
priate, recognized these negative factors 

1 Average mathematics rat score 

2 Low faiRli-whiiul rank 

3 Low Occupational Level score—"interests evidently imma¬ 
ture” 

4 Can mathematician key of Strong Tut 

5 Totally self-supporting 

C Low grades in high-scnool mathematics 

We thus have two counselors who considered and listed the 
same factors, and yet one decided the choice was inappropriate, 
while the other wrote’ 

The above is strong evidence to question his choice, but because 
of the ponlive factors which ire also dearly indicated, I d be inclined 
to give him the benefit of the try-out. 

The above counselor's note concerning the Occupational 
Level score gives a suggestion concerning one possible source 
of this disagreement This counselor evidently interprets tbe 
Occupational Level score as an expression of interest maturity 
which therefore might change as the student agea, while tbe 
first counselor vaguely interpret* it as an index ol academic 
motivation 

Only one of the counselors approving of the occupational 
choice mentioned the financial prohlem, and none of them em¬ 
phasized it as being important The anginal counselor recog¬ 
nized the financial problem in the interview and took steps to 
meet n Although he realized the importance of this problem, 
he did not consider that the boy should stay out of engineering 

This boy completed the first two quarters in the Institute of 
1 ethnology with an honor point ratio of 42. His grades were* 

College Algebra D 

Chemistry C, C 

Composition D, I (incomplr* * 

Drawing D, D 


He dropped out of school at the end of his second quarter 
and has not returned. 

Case number 13972 was an eighteen-year-old boy who grad¬ 
uated from high school six months before coming to the Testing 
Bureau and who was completing six months of post-graduate 
work m high school His occupational choice was mechanical 
engineering, hu father was crippled with arthritis and unable 
to work, he bad two younger sisters, sixteen and eleven years 
old, he had work experience drafting on an N YA job, and 
would be totally self-supporting. 

Test scores were as follows* 
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Strong Test scares were A on keys for chemist, production 
manager, farmer, carpenter, printer, mathematics and physical 
science teacher, accountant, office man, B+on keys for engineer, 
forest service man, B on keys for dentist, personnel manager; 
social science high-school teacher, purchasing agent. Aforew- 
ImityFeminimty percentile—76, Occupational Level porcentile 
— 22 . 

The following statement of factual information was ob¬ 
tained from data contained in the interview notes of the coun¬ 
selor who originally counseled the student 


The boy does not have much money. Hu father is disabled be¬ 
cause of arthritis. The boy has bean working part-nine for five 
months on a N Y.A job doing drafting work and making about 3100 
■ month. He has hod other odd joba from time to time. 

He is intrigued with mathematics and dqesn't know much about 
tho work of en actuary He docs not know that the training for 
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thia type of work is essentially through busmen and not through 
mathematics, 

He has applied for a scholarship open to eons and daughters of 
World War veterans 

He is socially re ruing. 

The original counselor and the case reviewer decided that 
the boy's choice of mechanical engineering was appropriate 
In making ihe later judgments, four of the five counselors 
agreed that this choice was appropriate, one decided it was not 

The dissenting counselor wrote* 

rhis it a loose judgment It would actually depend upon a judg¬ 
ment of which field (institute of Technology or actuarial) the boy 
wanted most He would be handicapped by lack of chemietry [in 
high school]. 

Three of the other counselors recognize the boy's expressed 
and measured interests m the business field and two of them 
suggest possible combinations of business and engineering. 
One of these two "mildly objects" that the boy hasn't quite the 
personality type for executive or management work. Two of 
the counselors indicate t&t the financial problem might serve 
m a basis for someone disagreeing with their judgment of the 
choice as appropriate Both say, however, they think the 
problem can be satisfactorily managed One counselor men¬ 
tions the low Occupational Level scores but thinks this is over¬ 
balanced by tho excellent achievement in high school Thia is 
the same counselor who m the last case interpreted the Occu¬ 
pational Level score as an index of maturity He is apparently 
using it here as an indicator of potential achievement! This 
counselor also recognizes the low score on the chemistry test 
but discounts it because the boy has not had chemistiy in high 
school 

In general, there are no fundamental disagreements among 
the counselors regarding this case They all tend to recognize 
that the boy has several potentialities and is a good college risk 
Although one counselor Indicates he would not encourage the 
occupational choice, he later qualifies this. 

This boy completed the first year in the Institute of Tech¬ 
nology with an honor point ratio of 2,11. Hit grades werei 
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Algebra A 

Trigonometry A 

Analytic Geomsny B 
Chemistry B, B, C 

Composition A, B, B 

Drawing B, B, C 

He is now making satisfactory progress in his engineering 
course 

Cose number 14095 was an eighteen-year-old boy who grad¬ 
uated from high school the same month he came to the Bureau. 
Hi* vocational choice wee buunsss, the occupation of hu father. 
Hu test scores were as follows. 
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Strong Test scores were* A on keys for personnel manager, 
real estate salesman; B+- on keys for social science high-school 
teacher, accountant, office man, sales manager, life insurance 
salesman, B on keys for production manager, Y.M.GA. secre- 
tary, purchasing agent Masculinity-F*tnsmnity percentile— 
76; Occupational Level percentile—66. Porcentile score on the 
Kuder Preference Record were: Scientific, 42, Computational, 
79; Musical, 10, Artistic, 7, Literary, 62, Social Service, 60; 
and Persuasive, 99 

The following statement of factual information was ob¬ 
tained from data contained in the interview notes of the coun¬ 
selor who originally counseled the student. 
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He u a young man of ihort stature, not too neatly droned, has i 
nther informal manner to the interview He la ratnu ordinary ia 
appearance. He would like to be self-aiipparnng while grant tn 
idiool He hai about $250 laved now and might earn enough before 
tuning at die TJiuveituy to cotiunue without working while m 
ichoob Hii father would help him but he diehkea Hiking hie lather 
fwhelp 

The father and mother were divorced about three year* ago, The 
lather u the ptemdent of a bus! new, There are no other children tn 
the family and the boy lives With his mother and get* along well with 
both parents. 

In lugh icfiool be uya he studied jiut enough to get by but «yi 
be realize* now chat that wu not eo imart 

The original counsel dt and the ewe reviewer decided that 
the boy'a choice of business was appropriate. In making the 
later judgment, all of the five counselors agreed that this choice 
was appropriate. Four ol the five said they did not think the 
boy’a chances of graduating from the Umvenrity were good but 
their attitude u expressed by this counselor. 

I cant lec any bun for disagreement mice the fob u waiting for 
him when he it through with school- However, I do not think htH 
graduate from the Umvenity 

Again in this case there is general agreement among the 
con melon. It is doubtful if the boy’s plans would have been 
Influenced differentially by the counselors as they were all 
ready to approve of his vocational choice. This case is quite 
clear-cut. 

This boy entered the College of Science, Literature and die 
Arts end at the end of his first quarter had an honor point ratio 
of .00, It is doubtful if his academic "flop 1 ' will seriously Inter¬ 
fere with hii vocational adjustment Hn grades werei 


Competition—cancelled 
Economics D 

Polidcil Science C 
History F 


He did not return to the University after his first quarter. 

Casa number 13835 was an eighteen-year-old boy who came 
to the Testing Bureau the month he graduated from high 
school, His father was a talesman. The boy had oAe younger 
sister The famdy was going lo finance hii college education 
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He had no work experience outside of working for his father 
and his vocational choice was advertising. 

The teat score* were* 
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Scores on the String Interest Test were, no A ratings, B+ 
on keys for farmer, purchasing agent, real estate salesman; 
B on keys for personnel manager, office man, president of a 
manufacturing concern, MascmknetyFembumty percentile— 
57] Occupational Laid percentile—40. Parcentild scores on 
the Kuder Preference Record were. Scientific, 57’, Computa¬ 
tional, 45, Musical, 1, Artistic, SB; Literary, 15, Social Service, 
17; Persuasive, 52 

The following statement of information was obtained from 
the dau contained in the interview notea of the counselor who 
originally counseled the student. 

He u a vary immature boy and hu mother cane with him the 
fint day He gives the impression of not knowing much about any¬ 
thing. He doesn’t react favorably to General Cbfiegn. 

The original counselor and the cose reviewer originally 
decided the boy's choice of advertising was inappropriate. In 
making later judgments, all of the five counselors agreed thst 
this choice was inappropriate Three of the j’udges could see no 
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reason why anyone would disagree with them, Two of the 
coumelon suggest he nught he able to handle a tower level job 
of advertising, 

They might think ho could do a low-grade tvp« of advertising 
(show card wntmg, window display*) which may ba what he means 
by advert*in man. . He could inner ret the degree from the 
idveniifot sequence in the Burney* School because of vsrkna rea¬ 
son!. I doubt whether lie’d be much rood even at a low level of 
advertising In view of the poor manipulative ikilli and ability to woik 
In spatial rdaocmahip*. 

The judges appeared to have veiy little difficulty in srnv- 
mg at a decision on this case. He entered the General College 
and at the end of the first year took three comprehensive ex¬ 
aminations, receiving a C in Social Civics, an F in Vocational 
Orientation, and an F in Individual Orientation. His course 
grades were*. 

Individual On on radon C 

Current History B C B 

Government Studies B B 

United States m World civilisation B C 

Vocational Orientation D 

Homan Bw)o|y C C 

Oral Communtunofi C 

_ S 

Human Development C 

Ecoaoadei C 

Care member 13964 was a seven teen-year-old boy who came 
tn the Bureau the month he graduated from high school. Hi* 
Tather was a car and truck mechanic and there were two 
younger brothers end a younger sister in the family He had 
bo unusual work experience and ba vocational choice was 
architecture. Test score* were a* follows: 
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Scores on the Interest Test were. A on keys for printer, 
mathematics and physical science teacher; B+ on keys for far¬ 
mer, personnel manager, accountant, office tnsn, B on keys for 
carpenter, forest service man, Y.M.GA. physical director, 
YJ4.CA. secretary, social science high-school teacher, Mas- 
cvknitp-Fenummty percentile—30; Occupational Level pir- 
centile—22 Percentile scores on the Kuder Preference Record 
were* Scientific, 50; Computational, 65; Musical, 63; Artistk^ 
90, Literary, 35, Social Service, 5; and Persuasive* 30 

The following statement was obtained from the data con¬ 
tained in tha interview notes of the counselor who originally 
interviewed the student. 


Than ii a financial problem here, to same extent. He hu ap¬ 
plied for N.Y.A. assistance He claims ht enjoys art very much and 
has done a great deal of poster work and layout work. He haa a 
chance for a scbolarahip sc the Inethutc of Art but will noc accept it. 

The original counselor and the com reviewer decided the r 
bay’s choice of architecture was inappropriate. In making the 
later judgments, two of the counselor* agreed that the choke 
was inappropriate, three decided it was not, Al! the counselors 
rating the choice as appropriate noticed the absence of raoa- b 
■ured interests hot made the following statements 

"No confidence in Strong for architecture/ 1 "Kuder test , 
■hows 'artistic*; haa art akilla, has Decenary ability. HieV 
Strong test may be in error." "The Strong ia against me bufu' 
hu experience (making posters) ia good enough to lead me cd 5 
let him try it Hie other interest teat supports hjs choice/* * 
One of the counselors rating the choice inappropriate cited 1 ? 
in support the great concentration of skilled trades interest^ 
the low Occupational Level score, the poor home and cultural 
background, jtnd the financial problem. The superior high? j 
school rank was discounted In light of the relatively poor schopl A 
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the boy attended The other counselor rating the choice as 
inappropriate admitted the boy probably had enough ability 
to get the degree and that the choice might be appropriate from 
rbe irandpomt or job opportunities He added, however. 

He probably 11 not outstanding enough in ability to make his liv¬ 
ing in this field, which offer* limited opportunities to ill but the very 
exceptional person In view of the lack o[ Strong s measured mtw- 
eiti, the low level ol occupational aspiration, Id i»v be should prob¬ 
ably get mechanical drafting mining or cammerqal art training. 

This counselor, who i« now interpreting the Occupational 
Level score as a measure of occupational aspiration, is the same 
one who previously interpreted it as a measure of interest ma¬ 
turity! Thu boy did not enter the University 

In this case, the basis for disagreement is quite clear-cut 
Two of the counselors, chiefly on the basis of the interest test, 
would discourage the boy from entering architecture The 
other three would discount the lack of measured interests and 
not discourage the boy. A tendency has also been found for 
some of the counselors to consider some of the keys of the 
Strong Test valid and some not. The factors which help the 
counselors determine which key* are valid have not been 
identified, 

Case number J40&1 was a nuieteen-year-old boy who came 
o the Bureau one year after graduating from high school He 
nil have to help finance hie college education His vocational 
hoice was metallurgical engineering 
The test scares were as follows 
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Score* on the Strong Test were: A on keys for engineer, 
chemist, Tanner; B+ on no keys, B on keys for architect, mathe¬ 
matician, printer, mathematics and physical science teacher, 
purchasing agent MatevUmiy-Femitmity percentile—65; 
Occupational Level percentile—66 Pereentile scores pn the 
Kuder Preference Record were* Scientific, 70; Computational, 
91; Musical, 50, Artistic, 55; Literary, 5; Social Service, 9; 
Persuasive, 60. 

The follenvuig statement of information was obtained from 
the data contained in the interview notes of the counselor who 
originally interviewed the student 

He live* with his mother and sister, both of whom are working. 
His father it dead. Hs applied for adnusaion last year but was re¬ 
ferred to the General College. He did not wwh to go there so hs did 
not come to school He has in uauaually strong vocational fixation. 

The original counselor and the case reviewer decided the 
boy's choice of metallurgical engineering was inappropriate 
In making the later judgments, three of the counselor* agreed 
with this decision while two counselors judged the choice as 
appropriate Two of the counselors rating the choice as inap¬ 
propriate said they would expect no disagreement. The other 
counielor rating it as inappropriate said that, in the light of 
the boy’s apparent strong drive for engineering, and "with the 
equivocal results on the Strong's," some might claim he should 
be given an opportunity in engineering. This counselor does 
not expound die mentioned equivocalities of the Strong Test 
He decides, however, that, without an interview, the above 
factors cannot be given much weight. Both counselors rating 
his choice se appropriate recognize the boy's limited abilities 
and one mentions his relatively weak background in mathe¬ 
matics and chemistry. One of the counselors writes* 

They would question his ability, His vocational choice has 
changed since he left high school That might help him to work to 
capacity Then I think he could ntalce it . I would improve his 
motivation, I hope, and if so, he'd make it He Jacks vwbabinc 
ikdli. Thii would not be auch a handicap to him m the Institute 
of Technology 
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This boy did not regiitcr in the Umvemty. 

In this case the basu for disagreement is again quite clear. 
The boy has interests congruent with his choice but abilities 
and a background that are not encouraging Ail the counselors 
apparently recognized thess factors, but three of them decided 
engineering was not a satisfactory choice, while two of them 
decided to take a chance, not place too much weight on the 
ability factor, depend on the interest and motivational factors, 
and let the boy try It. 

Summary of Analysis of Judgments 

The preceding discussion has covered much territory and 
many inferences hive been drawn. Inferences drawn from the 
six cases discussed and the fourteen remaining cases are not 
conclusions bared upon experimental results but are generaliza- 
lions based upon observations of only a few cases Often they 
are merely climeal hypotheses that should he investigated. 
Some of them are nothing more chan verbalizations of con¬ 
ditions generally known to exist but often not explicitly 
recognized 

L In miking judgments regarding the appropriateness of 
students’ vocational choices, crimed counselors agree with the 
original judgment 84 per cent of the time. The tetrachonc 
coefficient between judgment* msdo by different counselors 
is .86, 

2. The majority of five trained counselors agreed in making 
judgments regarding the appropriateness of students’ voca¬ 
tional choices upon the basis of the cose notes with the decisions 
of the counselor who originally counseled the student and the 
case reviewer 90 per cent of the time 

3. In fourteen out of fourteen cases, the counselor who 
originally counseled the student and judged the appropriate¬ 
ness of hu vocational choice at that time made the came judg¬ 
ment from the unidentified case folder several months later. 

4 Counselors tend to approve of vocational choices oftener 
than they dusppmve of them. 

5. The Occupational Level score has been interpreted as a 
measure of interest maturity, as a measure of motivation, as 
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an index of occupational aspiration, and as a cause for poor 
achievement in high school 

6 Some counselors are reluctant to have a student of high 
ability enter business even if other factors suggest the advis¬ 
ability of such a choice 

7. When both measured and expressed interests are in busi¬ 
ness, counselor* will seldom discourage a student from entering 
business. They may disagree on the level of training desired 

8. Inability and interest factors indicate success in an,occu¬ 
pation, counselor* tend to think meager informational back¬ 
grounds cgn be compensated for. 

9 The appropriateness of a student's choice cannot easily 
be agreed upon when the occupational requirements, duties, 
and status are poorly defined 

10 The greater the apparent contradiction between the 
student's test scares, die greater will be the disagreement re¬ 
garding the epproprieteness of hu vocations! choice 

11, Some counselors interpret more rigorously scores on 
some scales o! the Strong Test than they do scares on other 
scales 

12. Counselor! do generalize result! of the Strong Test to 
occupations for which there are no keys Often they disagree 
upon the probable pattern people m a given occupation would 
obtain. 

13 Lack of ability and absence of measured interest* in an 
occupation almost always lead counselors to disapprove of a 
student’s choice of that occupation 

14. If a student’s aptitudes are too low, a counselor will 
disapprove of his vocational choice even if he has measured 
interests in that occupation 

15. Relatively little attention was paid to personality test 
scores in judging the appropriateness of vocational choices. 
Much attention was given to tests of ability, information, inter¬ 
ests and high-school achievement. 

16. Interest tests are used in determining occupational 
areas; ability and information testa and high-school achieve¬ 
ment are used in determining levels of training. 

17. Counselor* make decisions regarding the appropriate- 
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ness of vocational choices based upon generalisations which are 
often without support. 

18. Sometimes when counselors consider the same evidence 
in making judgments on the appropriateness of vocational 
choices, they will not reach the same conelunon. They some¬ 
time* place different weights upon teats at different times, 
interpret them differently, and disagree with each other regard¬ 
ing these interpretations. 

Evaluation of these results might cause one to question if 
counseling baaed upon such judgments cnn be more than syste¬ 
matic guess-work. In light of the statistical results, however, 
it must bo concluded that regardless of the discrepancies m 
interpretations and regardless of some of the peculiar logic Used 
by the counselors, the final outcome* at which they arrive do 
agree. We must remember that in the above cases the coun¬ 
selor* were forced to make a decision. In many actual inter¬ 
views, the counselor can suggest try-outs of various kinds that 
1 provide additional information in tenna of actual success or 
failure before a decision ha* to be made. Often the counselor 
cannot make a decision and docs not have to put himself on a 
limb as he did here. The student is also an important factor in 
arriving at a final decision m the actual counseling situation. 
The responsibility for making the decision n eventually his, 
and counseling which is more "non-directive'’ in character may 
not force the counselor to make a decision at all As long aa 
the counselor is to present test data to the student, however, 
he will most likely have to interpret it as either supporting or 
not supporting the student's choice or else interpret it as being 
completely irrelevant. 
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THE DEFINITION OF SOCIAL EFFECTIVENESS AND 
LEADERSHIP THROUGH MEASUREMENT* 

HENRY C. LINE 
The PaycMagteal Corpora two 

Enough experimental work has now been done to pennft 
a definition of social effectiveness and leadership, and to justify 
such conclusions as the following (1) Social effectiveness cm- 
sists of certain habits and skills which can be acquired by 
practice, (2) Leadership u an aspect of social effectiveness, 
leaders are not merely bom, they are persona who have d*. 
valoped social effectiveness to an unusual degree, (3) Thu 
"only a few are bom to bo leaden, the great majority mint be 
followers,” is a fallacy—most people can develop and use ikifc 
of leadership if they will The baaia for these statements and,, 
a set of guides for the development of leadership and lodil? 
effectiveness will be presented here. 

Measurement 

The scientific definition and explanation of human tram 
depends upon the development of tests which will measure 
those traits The wide variety of measuring instruments de¬ 
veloped in recent years under the broad heading of personality < 
tests has contributed much to our understanding of soaiL 
effectiveness. These testa, or scales, or inventories, center 
around the concept of personality as the social-stimulus value, 
of a person Whereas recently there has been a trend to make 
the term, personality, synonymous with person or individu¬ 
ality, measurement has developed according to the more spe¬ 
cific concept of personality in terms of social effectiveness. 

* The piper was pm. a red far PSYCHOLOGY SCIENCE AND PRACTICE 
ednd by EL R Henry ind Daugfu H. Fryer, publiatmi sf which Ira Fimr ui 
Blnebirt hu been delayed by tna wtr Publication oT ibn piper ■ wui U 
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From the many excellent personality tents, the following 
have been selected to illustrate certain positive and negative 
aspects of social effective ness. The Moss Social Intelligence 
Test (1930), because it illustrate* the difference between social 
knowledge and social effectiveness; The Allport Ascendance- 
Subnussion Scale (1926), because it illustrate* b widespread 
belief ihat people fall into two broad classifications, those who 
dominate and those whq are dominated. The Vineland Social 
Maturity Seale (Doll, 1996), because more than any other test 
or inventory it exemplifies die concept □! social effectiveness as 
the acquisition of specific habits and habit patterns through an 
orderly process of learning, and the Link Personality Quotient 
Test (1935), because it is the only test the validity of which 
has yet been established in terms of leadership. 

Sodd Intelligence —Social intelligence is not synonymous 
with social effeetivenen An investigation of practically every 
experiment m which the results of personality testa were com¬ 
pared with those of intelligence tests, tests of academic achieve¬ 
ment, yean of education, or records of scholastic standing, 
reveal* the almost complete absence of significant correlation 
between them (Link and Roslow, unpublished), that is, mea¬ 
sures of social behavior do not give correlations with measures 
of intelligence or academic achievement. On the other hand, 
significant inter*correlauona are found among nearly all of the 
various tears of personality, such as at reversion-Introversion, 
ascendance-submission, sociability, sense of inferiority, emo¬ 
tional stability. This broad experimental result makes uwvi- 
I table the conclusion that personality twu measure a common . 
factor quite distinct from that measured by all forms of intelli¬ 
gence and academic achievement tests The I.Q. (intelligence 
quotient) or the A Q. (achievement quotient) measure* one 
aspect of individuality, the S.Q (soda! quotient) or the P Q 
(personality quotient) measures a quite different aspect 

This difference between intelligence and knowledge on the 
one hand, and habits of social behavior on the other, becomes 
dramatically dear with reference to the Spend Intelligence Test, 

This ttst correlate* not with the poreonalky tests but with the 
intelligence testa. This tort emphasizes an understanding or 
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knowledge of social behavior, whereas the emphasis in the AH- \ 
port A-S Scale, the Vineland Social Maturity Scale, the Link 
P Q. Tut, is upon specific social habits. The fonner stresses 
theory and judgment, the latter practice. Social effectiveness f: 
consists of a peculiar collection of habit* and skills, the common ' 
denominator of which is their practical effect on other peopled; 

Ascendance and Submission. —The A-S Scale codifies a con*;] 
copt of social effectiveness quite widely accepted both by sewn* - 
tific psychologists and the public According to this concept, ft 
some people dominate others in their social relationships, while i 
some are submissive, bashful, or retiring. This idea was pop* 
larly expressed in Scott's pioneer book on influencing men is \ 
f business (1911) and u also exemplified m another early books? 
by Scrong on the psychology of selling life insurance (1922). 

The ascendance-submission concept pf social effectiveness, ,’•» 
crystallized by the AS Scale, Is earned over into the BemJ^ 
renter Personality Inventory and a number of ocher modem ) 
personality scales 'The use of such scales implies chat a penon. j 
must either dominate others or be dominated by them. Tbit j 
fits in with tho popular fallacy that people naturally divide into g 
two groups,leaders and follower!. To be sure, test scores shaw|j 
many people at the happy medium, neither ascendant nor «Uh> $ 
missive. Nevertheless, rile implication is that both extremes^ 
are essential We ate here confronted by two aspects of psp'f 
chology, sometimes called pure and applied psychology. Obfll 
Is ths impersonal science of people as they are, the other i* th?»| 
science of people as they ought to be. According to tho findsjl 
many people simply are submissive and many are daminanfoj 
and these are the cold facta The second emphasizes the stuthl^ 
of people's capabilities and tries to establish standards or idewj| 
for there capacities. In the light of ouch teats and experimental'! 
a definition of social effectiveness in terms of chose who dorajfj 
nate and those who are dominated, those who Are leaden aiwp 
those who are not, i* inadequate- .$$ 

Social Maturity. —The Vineland Social Maturity ScaUmM 
unique in the extent to which it embodies the measurement^ 
social effectiveness as a result of learning. Its consttucd«p 
make* this clear Practically all of its 117 items refer to ir|| 
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cific habits and habit patterns graduated according to their 
complexity into 17 age periods All of these habits are of a 
bind which can be or roust be acquired by practice. The more 
of them the individual acquires, the more socially mature he 
becomes, no matter what his chronological age However, the 
grouping of habiu by ages implies that the normal person will 
acquire the habits characteristic at his age. There n no impli¬ 
cation of social dominance or submission, except in so far as the 
person who has acquired these habits well may be superior to 
the person who has not learned them. 

This scale u unique also ui the extent to which it consists 
exclusively of objective habit or behavior items, at least ui 
intern Unlike so many personality tests and scales, it asks no 
questions about how the individual feds, whether he is ever 
troubled with a sense of inferiority or whether his feelings are 
easily hurt. Social maturity is defined entirely in terms of 
overt behavior, and not in terms of ambiguous introspective 
raponses to subjective questions Through this scale Doll 
almost seems to say. “If a person will take care of his habits, 
his feelings will take care of themselves." If this wis not Doll’s 
intention, It certainly was that of the author of the P Q. Tut, 
in which the items are almost entirely habiu and activities, and 
where such subjective traits ai emotional stability, senso of 
inferiority, of secunty, and of social acceptance were deliber¬ 
ately emitted 

A person’s feelings or emotions as a result of social situations 
are unquestionably Important However, both psychologtcal 
theory and the trend in the development of tests and m clinical 
work justify the conclusion that feelings of social inadequacy 
leading to a general emotional instabdity are not the cause of 
•oaal impotence. They ore the result of failure to acquire 
adequate social skills Therefore, while the emotions of frus¬ 
tration may give us broad duet to the problems of social compe¬ 
tence, the achievement of social effectiveness must be defined 
ID terms of the habits and skills which make it possible. 

Doll’s Seale is not divided Into traits, as are many person¬ 
ality tests. Nevertheless, some of the items in his scale include 
such a complex range of habits that they might almost be said 
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well have been given some such name as Social Effectiveness 
Quotient Tut, since it u concerned entirely with habhj and 
■kills in this category. These habits are classified in four groups 
in the test, representing the following four craiut (1) habits of 
social Initiative, (2) habiu of self-determination, (3) habits 
of economic self-reliance, (4) habits oT adjustment to the oppo¬ 
site sex. The title, P Q. Ttn, was chosen partly to emphasize 
the fact that these habits are the result of learning, and partly 
to emphasize the sharp consist with the I.Q. Experiments 
with many personality tens, os well as with the P.Q Tut, have 
shown that the habiu of an effective personality or a socially 
effective person were in a category quite distinct from the 
habiu of scholastic intelligence and achievement. This fact 
was confirmed in the nation-wide experiment to validate the 
P.Q. Tut (3,8). 

Leadership U a manifestation of social effectiveness Thu 
was assumed when the PQ. Tut was validated with refer* 
enc« to tho criterion, leadership. Leaders, for the purpose of 
the experiment, were high-school students who had bean elected 
by their classmates to positions oF leadership, not leaders ap* 
pouted by teachers or school officials They were the class 
officers, officers of the General Organization and other school 
societies, editors and managers of the school papers, and cap- 
taint and managers of athletic teams, school band and orchestra, 
glee dub, and other gimps. The non-leaders were those who 
held none of the positions of leadership, The leadergroup was 
found to have a significantly higher P Q. than the non-leader¬ 
ship group. Therefore, the P Q Tart measures social effective¬ 
ness in terms of leadership. 

Leadership 

Thu concept of social effectiveness which we have been 
developing came* us a step beyond that of social maturity in 
terma of the status quo. It introduces a qualitative factor, 
namely! what habiu and eltiHs contribute more than others 
toward social competence and leadership? Accordingly, much 
of the experimental work with the P.Q. Tut has concerned 
itself with item analyses and evaluations of the various habiu 
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to represent a unit trait. For example, the item, "assumes 
personal responsibility, u may well include such a net-work of 
habiu as would justify the trait description of self-reliance. 
In fact, most of the tnuu which other teau make explicit, plus 
a good many more, are implicit in the items of the Social 
Maturity Seale. 

There is one important exception, and that is the collection 
of habits which have to do with adjustment to the opposite sex. 
Nowhere m thu scale is the specific and distinctive character 
of these babiu recognized. On the other hand, clear recogni¬ 
tion is given to the importance of habiu and skills which con¬ 
tribute to social effectiveness by enabling a person to be inde¬ 
pendent of people, to get along without them. In children this 
consiiu of being able to dress alone, to eat without help, to go 
places alone In adults it consists of such items as "creating 
own opportunities” and “systematizing own work.” The im¬ 
portance of such habits in terms of social effectiveness is so 
obvious that it is easily overlooked For instance, how effec¬ 
tive, socially, could a healthy adult be who had never learned 
to dress himself? On a higher level, this fact is expressed by 
Emerson's epigram. 4 "The test of true fnendship Is the ability 
to do without it” A person who is entirely dependent on hu 
friends, like an adult who has never,learned how to get along 
without his mother, is limited in his social effectiveness. 

The person interested primarily in the practical problem 
of becoming more effective socially would do well to use the 
Sactal Maturity Seale as a guide. From the standpoint of 
scientific psychology some of its details are open to criticism. 
Nevertheless, it represents one of the soundest actemptt to 
measure social effectiveness thus far The next great scientific 
task is that of delineating the habiu of social effectiveness with 
a curriculum and senes of texts for instruction comparable to 
those in tb e academic field today Doll’s Social Maturity Seal§ 
points the direction of this development. 

The P Q -—The Personality Quotient Tut was constructed 
to express the definition that* "Personality is measured by the 
extent to which the individual has acquired habits and afc«H» 
which interest and serve other people" (3). It might just as 
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and activities. The following guides for the development of 
leadership and social effectiveness have been deduced from 
these and other Investigations 

Social Contact ,—Social effectiveness cannot be developed 
m a social vacuum. One learns the skills of serving others by 
working and playing with them, not by remaining aloof. Per¬ 
sons who take part m community affaire, or who have been 
aetive members of a Y.M.GA. or Y.W.CA, dramatic club, the 
ScouU, or other organized groups, were found to have a higher 
P.Q. than those who had not. The frequent and often sneering 
rejection of group activities is eloquent proof that individuals 
can deliberately choose not to develop the skills of leadership 
Parent! who do not believe in letting their children do a^they 
please, but who, by example and discipline, influence them to 
join the proper groups, are at least beginning their children on 
the path toward leadership. 

Organized Groups ,—Highly organized and ‘competitive 
groups do more to develop leadership than do loosely organized 
or casual groups. The habiu of teamwork and good sports¬ 
manship are the very foundations of social competence and 
leadership One question of the P.Q Tut is< "Which of the 
following have you engaged in frequently in season,—basebgll, 
tennis, bridge, badminton, hockey (a bat of 20) f" Those who 
had engaged in 6 or 7 on thu list usually had a higher P Q. than 
those who had participated in only 1 or 2. Tho next question 
is: "In which of these did you practice herd and regularly so 
that you would become good enough to enter a contest?" Such 
practice contributes even more toward a high P Q. than does 
casual participation Among young people the greatest angle 
influence in tho development of leadership was intensive paff- 
tlapstioo in competitive games and sports—however, not in 
one but in several. 

Prominent in this list are orchestras, glee dubs, and chona. 
Few activities require more intensive practice and team-work 
than an orchestra Not only must the individual learn to play 
hu own instrument adequately, he must leant to play it in 
harmony with twenty or thirty others (hie sour note may 
destroy the entire effect The best violin becomes the first 
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■Vlrtin, a leader, u! every competent pUyee may become the 
leader attmel with a Kilo - 

Pnvau Praeirec—The practice oi some iDcrel eloUi m 
private ia eaaentiel to Ae development ol leadetabip. The per¬ 
son who ipcnde day after day praetrcuig Ae piano in aolitudc 
te developing a ekill which may tranatam lell-diitniat mto 
eocral confidence, Almoat any akin which one developa to Ae 
point oi competence may give him ot hot a eenie ed confidence 
and a degree of leadenhip. Nor doee it havo to be a akall m 
mueic, art, Othterature. The mote homely ara, each a. cook¬ 
ing; knitting, and aewrng, telling etonoi, gatdmrog, being handy 
with tool!, if developed to Ao pant oi illpenonty, can aU ran- 
trihute toward racial effectrveneea, even to leiderahip. Nrarty 
every parson can acquire soma skill to the point when he excels 
m the ability to interest and «rvo where 

Economic Independence.—V* development of ImMcisl 
independence is indispensable to leadenhip The habits oi 
dDiner socially useful work, as represented by the tnit of eco- 
nomic aelE-rthance in the P.0. Tut. am of bum importance 
Aiidolromthe crpetiaitntal and dnaeal mdanra. lot th» ran- 
dnaion. It aaema only natural that the pardon tvho leadl othara 
and upon whom others depend, should ht independent wvd self- 

Sffw.-—Soaal effeettvenas includes habits of deal- 1 
m. with tbs opposite sex. Psychologies! practice shows that 
’ there are some individuals who live in only one half ot the social 
world, either the world of men or the world of women, they 
have developed emotional and mental barrier* which keep them 
from dealing effectively with member* of the opposite se* even 
in the moat simple social situations. Such a barrier u Wmly 
to exist whenever an individual baa fsDed to develop, through 
practice, some ol the elementary habits of pUjnng or working 
with members of the opposite sex. Many of thwe hrius, like 
aocial danang, are obviously of a very specific tort. However, 
even walking with, talking with, and telephoning to members 
of the opposite sex am found tn be specific habits which muat 
be and can be cultivated by themselves Specific habits of 
interesting and serving members of the opposite sex are sharply 
defined elements in tocW effectiveness 


Bodily Movement —Bodily movement w the common fo. 
nominator of the .activities which develop social effeenvenau, 

If one examines the activities mentioned above, or the items 
which make up the Social Maturity Scale and P Q, Tut, it ml 
be seen that all of them, whether it bo playing tho piano alone, 
playing m an orchestra, working for pay, or playing in a bue- 
ball game, involves physical movement The veiy word, 
loads r, implies movement and action The leader u one who 
moves in advance of his followers, one who must make unuiuil 
exertions in their behalf In terms of our definition of social 
effectiveness, he has won his position of leadership by his ability 
to interest and serve others. In so far as he has become flu 
servant of all he may become the master of all. 

The common denominator of those purauita which hinder 
the development of leadenhip is the absence of bodily move¬ 
ment, the absence of creative exertion. In this classification 
fall those time-consuming pursuits in which the individual i 
merely sits and takes things in—-the movies, the radio, caiusl t 
reading, and just plain sitting. j 

Lead-led .—Leaders must leam to be good followers most .1 
of the time. The condition of becoming a leader » the acqui-' 4 
•ition of skills which make one a good follower, a good team- j 
mate, For every situation In which a person acquires that | 
skills to the point of leadenhip, there are many more situitioni 1 
in which his social effectiveness depends on the skills of bem| j 
a good follower A study of leaders and non-lcadere (10) u 4 
which 40 outstanding leaden and 40 non-leadere were selected j! 
ui each of three high schools showed that the leaden p»nia- ' 
pated in an average of 6 ft activities compared with 1.7 activi-' 
ties for the non-leaden The most frequent extra-curricular * 
activities among leaden were competitive athlericB Moreover, , 
the leaden had over twice as much spare time as did the non- 
leaden, in spite or the fact that they engaged in four tunes ii 
many extra-curricular activitiea The authon attributed this 
to the fact that non-leaden frittered away much of their time 
and spent more time in reading and solitary activities whereas 
the leaden used all their time in a better organized and more 
efficient fashion. The participation in highly organized nctW- 
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ties had brought about a greater integration inrtheir lives gen¬ 
erally. “Tho fact that the loader* participate in gamee and in 
attending parties,* say the authors at wve point, “is highly sig¬ 
nificant In games the leader guns more exponents ^with 
people, and thL experience is useful to him in leadership ” 
Especially Interesting is the light east by this study on the 
oft-made remark* "We esmrot all be leaden, some of in must 
be followers p This study showed that the leaders wore also 
the good followers, wheresa the non-leaden were the poor fol¬ 
lowers. That vs, a leader ought be captain or top mao in cos 
activity or two, but was merely a member of the team or squad 
in his other 5 or 6 activities In other words, the leaden were 
really folowen in 5 or 6 activities, whereas the non-leaden were 
follower! in gniy 1 or 2, or none. Thus, conspicuous leadenhip 
in one activity was based on having become a well-disciplined 
follower In several activities. The individuals chosen by their 
fellows to be leader* were tolerated as leaders because they were 
good follower*. 

C*m«uiip 

The critical problems of citizenship so sharply illustrated by 
current perplexities over juvenile delinquency give the above 
findings unusual importance In so Tat as good citizenship can 
be developed through education, these finding demonstrate the 
necessity of an almost revolutionary change in the present edu¬ 
cational system. This change may be described briefly by say¬ 
ing that many of the compulsory elements of education should 
be made optional and many of the optional elements made com¬ 
pulsory, tLt is, the extra-curricular organised group activities 
should be made partof the regular curriculum for which credit 
Is given, and many of the present required subjects made cXtra- 
eatricuUr, 

Intelligence and knowledge are necessary but, as we have 
ifeen, are by no mean* sufficient In fact, there is no correlation 
i ( today between sqcisl intelligence and social effectiveness 
I JEffacthre dtiaensbip depends on habits of action m organised 

groupt* where mxt lodmflwla ftt* good follower* most of the 
dips' but alsp good leaden *pme of the time* The individual* 
* whd Irani the habits of good citnenahip in the smaller unit* 
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such as those enumerated under the heading, organized group!, 
thereby acquire tho basic thills required for effective partuaps- 
tion in tho larger units of a democracy. 

REFERENCES 

1. Allport, G W and Allport. F, H. The A-S Reaction Study. | 
Boston Hougbum-Miffiin. 192ft. , _ ,, 

2 Dell, B A. "Annotated Bibliography on tho Vineland Bodd 
faieuiitv SajU." Journal of CownWeg Psychology, IV 
(1940) 123-132, ' 

3, link, H. C. ' The Rediscovery of Man. New York MicroUJm, 
1937, Pp 257 Chapters 3 and 4. "The Piychodynsnua 
of Personality and the HaW» erf Personality ,"pr 55-90. 

^ s'i) . < *52«^" nt *-. 

S Lmk,HC. M.rira! ft* Uu P.0, w Pmo«^J.ay Oueumi T>rt, 

New Yorit. T1wFaydiolope*l Corp«.ta». 

6 . loekhirt, E. <3 Imtnvt*, ruuPumdily, Lo* Anpk. , 

W.lron Plibluhin, Co. 1939 Pp. 513 , 

8. RoiloW^^^Jarira^WUe^and Local Volidadon of riaPft*, 

Pereoiulity Quotient Test" Journal of Applied PjyeW- 
ort,»OV D940),S19-S39 „ . , ,> 

9. 5ratt, W. D in 

Arcwnraiewd Suggtttue. New Yotk, Repaid Press, ml, 

10. SmiA M^nd N*.trrm, W. C. -A Study of Sodal Part!*,; | 

txm and of Leisure Time of Loaders and Non-Leaders. 

i» Edvcntwual lusMul icw, diap VII, 197-22S, TWrt^ 
Seventh Yearbook of the Nadoaal Soaery tor the Study 
Education. Part L Bloenangton, 10/ rubnc Sebom 

12. Whlre^f! ^ Dealing unth People, thpfi 



70 EDUCATIONAL ADD TflVCHOLOCICAL MBAADEEMSIfT 


A REPORT ON SCHOLARSHIP EXAMINATIONS 
GIVEN IN LATIN AMERICAN COUNTRIES 
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In August, 1942, work mi begun on the planning of an 
Institute of Meteorology in Colombia under the joint auspice* 
of the United Sute* Weather Bureau, the Office of Inter-Ameri¬ 
can Affsin, the Defense Supplies Corporation, and the Division 
of Cultural Relations of the State Department. 

The Institute was to award scholarships to 200 Latin 
American young men for training in Weather Observations, 
primarily, preparing them for position* in the Tnercorologjcil 
services of their own countries or of Latin American Airlines. 
Courses in meteorology, and refresher courses in physics, 
mathematics, and English were alio to be presented The 
Institute wa* to be staffed partly by U. S. Weather Bureau 
personnel and partly by Latin American instructors 

The long-range objectives of the Institute of Meteorology 
in Colombia were threefold. 

*1 To increase the safety and regularity of air transpor¬ 
tation. 

"2, To pave the way for improvement* m agriculture 
through better crop planning and forecasting. 

"3. To provide greater facilities for flood forecasting, pub¬ 
lic health work, and Improved operation of public 
utilities through advance knowledge of weather ” 

In the planning of the program the question inevitably aiwe 
a* to how young La tin-American men were to be selected for 


the training in the Institute The men selected had to meet 
the following requirements 

"l. They must have a knowledge of physics and mathe¬ 
matics, or they must at least show an aptitude for 
the study of physical science. 

“2 They must have a knowledge of English, or at least 
have an aptitude for learning languages " 

The requirement of a knowledge of English arose from a 
further aspect of the planning of the Institute. After the Bin- 
month course or the Institute was completed, 40 of the better 
qualified students were to be brought to the United States and 
provided with advanced meteorological training at one of our 
universities, Thus, though it was not necessary to select only 
those who had a good knowledge of English, it was desirable to 
have as many so qualified as possible, from whom 40 might 
later be chosen for advanced training in the United States. 

The young men were to be selected from each of the Latin 
American countries, the number from each country being pro¬ 
rated according to population, area, and number of airline- 
miles 

The first method of selection that presented itself was to 
bate selection on achievement in physics, mathematics, and 
English, as shown by school records It was immediately ap¬ 
parent, however, that the scholastic standards of each country 
are different, and, furthermore, that an evaluation of the 
courses of study even within each country would be extremely 

difficult. *' 

It was soon decided that in order to secure objectivity in 
the selection process, it would be necessary to give the candi¬ 
dates an examination which would measure such factor* as 
knowledge or elementaiy physics, knowledge of English, and 
aptitude in physical science and language The United States 
Choi Service Commission was asked to design such a battery 
af tests, co be translated inco Spanish, Portuguese, and French 

The examination was scheduled December 15 and 16,1942, 
in the capital cities of the various Loan American nations. It 
was open to individuals between 20 and 30 years oF age "who 
have completed the usual academic preparation for entrance 
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to the national university of their own Qr a neighboring 
country. 1 * 

The written examination consisted of 4 objective tests com¬ 
posed of multiple choice items Test 1 was a mathematics and 
physics test composed of eighty items, as follows: 35 mathe¬ 
matical problems in physids, 25 phyiics information items, and 
20 mathematics items. The 20 mathematics items consisted 
of 9 geometry, 5 algebra* 2 tngonomctiy, 2 analytic geometry, 
and 2 calculus items. Test 2 was an Engluh test composed of 
100 items, os follows; thirty-live vocabulary Items, fifteen 
grammar item*, and fifty reading-comprchenaion items. Test 
3, a language aptitude test, was composed of 150 items, as fol¬ 
lows 1 forty items requiring the competitor co form words from 
jumbled letters, eighty items requiring the deciphering of coded 
words, and thirty items requiring the translation of an artificial 
language. Test 4, a physical-sciences aptitude test* consisted 
of 100 items, as follows. 2S spatial-ability items, 25 items re¬ 
quiring the competitor* to predict the motion of various illus¬ 
trated mechanical devices, 25 items requiring the insertion of 
a musing number in a number series, and 25 items requiring the 
competitors to discover the differences m a set of similar geo¬ 
metrical diagram*. 

The examination was printed in Portuguese for sdmmistra- 
, turn in Brazil, m French, for administration in Haiti, and In 
Spanish, for administration in the remaining Latin American 
nations. The examination scores of 708 competitors were avail¬ 
able lor statu tic al analysis The score on a single test was 
obtained by dividing the number of correct answers by the 
Dumber of items in the ten and multiplying by 100, the total 
score was obtained by adding together the number of correct 
answer* on the Tour teats, dividing by the total number of 
item*, and multiplying by 100. 

Fiwfengx 

Table 1 shows the number of cater, the meant, and the 
standard deviations of total score* for the three translation! 
of die examination. It ii to be noted that a companion of the 
mean scores of the Haitian competitors, the Brazilian comperi- 
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ton, and the competitors from the Spanish-speaking nations 
provides no basis for concluding that the three translations of 
the aamination differ sharply in difficulty from each other. 

Tabic 2 summarizes the obtained date relevant to the effec¬ 
tiveness of the 4 testa as selective devices. The sample of 185 
cues upon which the calculations were based was obtained by 
selecting every third paper from the papen of the competitor* 
who took the Spanish translation of the examination 

The figures of greatest interest in Table 2 are the estimated 
reliabilities of the four teats The high estimated reliability of 
the Engluh Test can be quite plausibly explained on the reason¬ 
able assumption that the range ui the competitors* knowledge 
of English was very great, as indicated by the fact that scores 
on the test ranged from 0 to 96 

The extremely high estimated reliability of the Language 
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Aptitude Test cannot be attributed primarily to the Wh «C 
tha tm, since reduction of the number of items Irom 150 to 100 
would bo expected to reduce the reliability of the teat only by 
about 0.01, nor is it plauwbla to wume that the range of tire 
language aptitude of the competitors was unusually great. «* 
spite of the fact that Bcores on the test ranged from 2 to 95, 
Since presumably the range m language aptitude among the 
competitors u roughly the same as would be found among a 
group of high-school graduates in the United States One 
plausible explanation that suggests itself is that many of the 
competitors may have completely faded to understand the 
directions of one or more of the 4 different types of items con¬ 
tained m the test ir this actually occurred to a large extent, 
it would be expected that the test would have high 
as a result of tho fact that the test would function as a hlgWy 
reliable measure of the competitors' abihty to understand the 
particular directions contained in the test. Such a test, how- 
ever, would be undeairahle for tho present purpose since not 
only would the test be a leas effective measure of language apti¬ 
tude than it should be, but also .t would not even be a good 
measure of ability to understand directions other than that 
particular sot of directions contained in the test. 

The estimated reliabilities of the Mathematics and Physics 
Test and of the Physical-Science Aptitude Test are not so high 
BC would be desirable In the cam of both there tests the low 
estimated reliability ia presumably attributable to the fact that 
both tests appear to have been very difficult for the compote 
tore. The ranges of scores on the two tests are from 0 to 70 
and from 6 to 74, respectively- For the present purposes, the 
low reliability" of these teats are not so serious as might at first 
appear, since recent developments m test theory indicate that 
m the case of a vety difficult test the scores of the higher-scoring 
competitors are probably more reliable chan would be indicated 
by tho reliability coefficient calculated from the Mores of the 
enute group of competitors 

The intercorrelaiians of the tats ate not so high, as to indi¬ 
cate • serious degree of duplication among the testa A* might 
have been anticipated, the correlations of the English Test with 


THE 1943-1944 WORK-SESSION OF THE COUNCIL OF 
GUIDANCE AND PERSONNEL ASSOCIATIONS 

T«B second Work-Session of the Council of Guidance and 
Personnel Associations was held at the Hotel Biltmore, New 
York, on Thursday, November eighteenth The meetings 
were scheduled (or November instead of Januaiy with the 
thought that the representatives of the constituent organa*- 
pons might profit from an earlier discussion of current problems. 

The morning session was largely devoted to the general 
subject of Rehabilitation. 

"Summary of Findings of 1942 Meeting with Emphasis on 
Changes during 1943" 

Lieutenant C. Gilbert Wrenn, Bureau of Naval Per¬ 
sonnel 

“Rehabilitation of Service Men and 'Women'* 

Mr. William Gilchrist, Chief of Rehabilitations, Vet¬ 
erans Bureau 

Dr Harry A, Jager, Chief, Occupational Information 
and Guidance Service, United States Office of 
Education 

“Rehabilitation of Working Students" 

Miss Beatrice McConnell, Director, Industrial Divi¬ 
sion, United States Department of Labor, Chil¬ 
dren's Bureau 

Gilbert Wrenn gave a brief hut very complete summsty of the 
work-session of lost January. He pointed out that we are now 
living out the plans discussed at the lost meeting; and we are 
now anticipating the post-war problems. Mr GDchriit re¬ 
viewed government procedure as it is at present in Operation. 

Dr. Jager spoke for the educational program that »just now 
getting under way. Mias McConnell reported briefly on the 
dislocation of the school population and the inevitable conic- p 
guences which will some day have to be dealt with 


74 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 

the other three testa am the lowest of the obtained correlations, ! 
avenging roughly 0J5. The intereorrelatione of the other 
three tests are all close to 030, indicating an appreciable, but 
not too senoiu, degree of overlapping 


-Although Latin Americans have limited experience with 
objective teats, since most of their school examinations are of 
the essay type, the Latin American officials and the compete- 
tom were in general highly pleased that the award of the 
scholarships was made on the basip of merit determined by 
objective tests, rather than by personal selection On the whole 
the project appenn to have been highly satisfactory faun the 
point of view of all concerned 
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The afternoon session emphasized the need for the “Preseiv- 
virion of Social Values in a Time of War " The speaken wpje 
Mir Robert P. Lane, Executive Director, Welfare Council o(► 
New York, and Dr Harry D. Gideons*, President, Brooklyn 
College Mr. Lane's speech was a philosophic approach to the 
social values in our civilization, President Gideonse discoursed 
on the social values which our educational system has tended to 
emphasize 

At the dinner meeting the speaker was Dr Luther Gulick, 
Chief of tho Planning Division, Office of Foreign Relief and' 
Rehabilitation Dr. Gulick had Just come from the meeting! 1 
fn Atlantic City and apoke on '‘Rehabilitation Abroad." - 
On Friday the Executive Council of the American College 
Personnel Association held its annual meeting. The members' 
present were E. G Williamson, President; Gilbert Wrenn, Vice- 
President, D D Feder, Secretary, Thelma MiUa and Helen' 
Voorhees Other member! in attendance were A. J Brum- , 
biugh, John G Dariey, Winifred Hausam, Forrest Kirkpatrick, t 
James Mcdintock and Robert Moore. * 

The necessary business was transacted and then the groups 
discussed at length some of the problems of post-war coujikL^ 
ling. The results of our deliberation will be reported to you is^ 
due time. J 

Helen VooWTipj ^ 
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AN INVENTORY OF STUDENTS’ GENERAL 
GOALS IN LIFE 


HAROLD B DUNKEL* 

Cm pm' it o Study k Gian) Educilnn 


What basic beliefs notv constitute college icudentB* "philos¬ 
ophies of life” or "designs for living"? Whac does the College 
student consider the main goals of his life? Far the sake of 
what values docs he think that he lives? To what extent does 
hu total pattern of values seem to meet certain criteria of a 
"good" design for living? The attempt to answer these and 
similar questions led to the project in philosophy of life and 
religion undertaken by the Cooperative Study in General Edu¬ 
cation 1 Almost without exception the colleges stated some 
such institutional objective as “to aid the student in developing 
a useful and desirable philosophy of life," and faculty members 
insisted that their institutions did not merely state this ob¬ 
jective in the catalogues but were sincerely striving by various 
means to aid the student ui developing an adequate personal 
philosophy 

Four passible steps in dealing with the problem of students’ 
philosophies of life can be indicated by the following questions 
I Do and should students have philosophies of life? II, If so. 


* Work on dill project Hi, been nittcd on hr faculty aumben of collegia par- 
tawatMf m d* Cnopwatiaa Study la Otnuil Education and by nwmhnm ci the 
nopal ial wbof ui the Sdd of Hmnimlus, Gtart« K. Barron, jr, Walker IL 
HB, and tba author Dr Barton, now Lt Barton of the U S. Air Cnoa, directed 
tfia original oigionition of liic project and tfw prtnmoM of ihc finr forro of ifio 
kvcalory Tha coinplata upon on tha project, rocludini nonru and data on idii- 
bfliry and yilrdity, a beam publafyid ikn aooimer 

1 Tha Cooparatm Study la an ornnlmron n ipprouunidy a acorn of coUegea 
who atmk coopcraiivdy, with (hr aid of a central raff, ihnr common educational 
proMarn*. Tot ■ tap, tt of the Study’s work. He Ralph W Oein, The Coopanthn 
Btojdy in Geacral Ed .cum," TU SJuti vm«f Jtarord, XXIli (October, 19 * 2 ), 
^'ira'iod “> sreater detail iboat the development of thu projert, 


--—- ---- -- k Rtratcr detail ibtwt the developrocnl of thu prolert, 
itw Stst Ntwt utter {The Cooperatm Study in Genaul Education. 
5WS Kkibuk At*, Qucaps, minoia), vet II, no 6 


what are these philosophies? Ill Once we have ascertained a 
student's philosophy, do we (the student and the faculty) con¬ 
sider it satisfactory, and on the basis of what catena do we 
decide? IV If a student's philosophy appears unsatisfactory, 
how can the college assist the student in working out a satis¬ 
factory view of life? 

Our colleges believed thzc the anfcwer to the first question 
was “Yes,” that students should have a philosophy. Many 
teachers and institutions had made a practice of having stu¬ 
dents wnte essays setting forth their philosophies Since these 
essays had all the limitations of any verbal statement (they 
are what the student thinks, or wishes to think, or wishes the 
reader to think, is his philosophy), these teachers were of 
course not under the misapprehension that there is a perfect 
correspondence between the way men live, and the way men 
say they would like to live The hypocrite, the person with 
selfish or antt-social aims which he fean to express, the victim 
of social or economic pressure, alL undeniably exist Those 
undertaking the project believed, however, that it is equally 
incorrect to assert that them is absolutely no relation between 
the way in which people talk about life and the way In which 
they live. The group felt that in many cases the verbal state¬ 
ment about the kind of life the student was seeking would be a 
useful index, even though not a perfect one, of the kind of life 
tho student Was living or seeking to live. 

In the case of many students, when there is a discrepancy 
between stated beliefs and conduct, this inconsistency exists 
without the person’s being aware of il For these students, an 
opportunity to state their beliefs precisely and to examine the 
implications of those beliefs for action may often load the 
student to harmonize more closely his pattern of beliefs and 
Ilia pattern of living. 

Other students actually do not know what they believe. 
They have never thought much about their way of life or about 
what they believe- Having acquired their schemes of value 
rather unconsciously and at random from various sources, these 
students, when they leave the pattern of life into which they 
were bom and grew up, and when on coming to college they 
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meet new and perhaps conflicting values represented by new 
environment, new friend^ and new experiences, have consider¬ 
able difficulty in ordering Aeir personal lives for the first time. 

Without help in seeing and resolving some of this conflict 
and confusion, the student may go through much if not all of 
his adult life, a victim rather than a master of these wamng 
values The puipose of the proposed device was, therefore, to 
enable the student to see clearly what he saya he believes, to 
compare it with the philosophies of other students, and to ob¬ 
tain help in overcoming difficulties and conflicts in hu own 
position. 

Tho group did not believe that all these results could be 
satisfactorily produced at the-verbal level alone. Even on the 
verbal level, we certainly did not wish that a student should 
work out for himself, once and for all, an immutable philosophy 
of life We believed, however that 


most mdien , mil agree that no student should merely drift 
through life, allowing hu major decisions and actions to he determined 
wr him entirely by circumstances—to he merely the resultant of 
external fortes impinging upon him Each student should bring to 
ecary important Die decision some "sense of values” which u hu own 
. » Of count such a "design Tor living" can never be completely 
npueit . . In a brief, oversimplified, presents non It (the idea 
of a designJ sounds unduly rationalistic—u if each student were ex- 
pectcd to chart all the details of hu life, to be conscious of all the 
act j “d to refer all decisions to some grandiose 
nodem of life and the universe We do not mean this» 


Rather the philosophy jhould be a "working hypothesis' 1 for 
imng, to be revised as and if necessary, in the light of new situ¬ 
ations and insights Though a satisfactory verbal statement 
of a philosophy or life is not the whole journey, it is at least a 
step in the right direction. Such reasoning as this, then, led 
our group to the belief that students should have a philosophy 
of Ufa (the group's answer to the question of Step I) and to an 
interest in securing a verbal statement of this philosophy (Step 

A further^result of conferences between faculties and staff 
^ dwkKm that some objective means of securing and re- 

*** fftmtUiUr, YflL II, da. 6, p 6. 


curding the student’s statement of hu philosophy was desirable. 
Although, to be sure, the response m the form of an essay gives 
certain insights which probably cannot ba made available 
through objective devices, the existence of the objective in¬ 
strument would not preclude the use of the e*sgy 

On the other hand, the objective instrument Would have, 
the group fclr, several advantages In the first place, the 
philosophies” thus recorded would be more exactly and more 
easily comparable. For example, the score of one student could 
then be compered to his own sporep at other points in his edu¬ 
cational career, to the scores of other individuals, and to the 
scores of groups. Similarly, scores or sections, classes, or entire 
student-bodies could be compared, and if desired, judged. 
Likewise the phdosophy of a student who, when left to himself, 
deals in any generalities would have more points of comparison 
with that of a student who plunges into petty mlnutlaej and 
the statement of the verbally facile student could more mean¬ 
ingfully be compared with that of one less articulate. 

Furthermore, the records of these philosophies would be 
more concise and more usable. Faculty-members who wished 
to feam about the philosophy of a group of students would no 
lpnger face the difficult end often impossible task of reading 
through hundreds of pages of student manuscript. Rather, 
the results stated numerically would permit fairly rapid sum¬ 
marization end comparison. Then finally, instead of existing 
like the essay in a single copy tucked away in some personnel- 
folder or teacher’s files, copies of die results of an objective 
record could be made available for study and consultation, 
easily and inexpensively, to every person concerned with the 
student For these reasons, then, the preparation of some ob¬ 
jective device of this sort seemed eminently dsmble as a first 
step 

Many aspects of the student’s philosophy of life deserved 
study; hut, for various reasons which need not be discussed 
here the group began work by studying the main goals which 
students hold. Hence, though other studies were planned to 
supplement the information gained^ from this device, the fint- 
mstrument which the group undertook to construct was the 
Inventory of General Goals of Life," 
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First, a list of possible main goals was secured from papers 
written by students and from teacher* in the group. In the 
selection of the goals finally included in the mventoty, two 
criteria were dominant The first was that the goals listed, 
though couched in student phraseology, should give an ade¬ 
quate representation cF certain of the great historical traditions 
of philosophy and religion The second was that the lut should 
also include goals which, though less common in formal philos¬ 
ophy and religion, are familiar in our culture or in "cracker- 
barrel philosophy" Within the limitations imposed by prac¬ 
tical conditions, we wished each student to find expressions 
which he could cdnndcr adequate statements of elements of his 
own point of view. 

Ilie number of possible "philosophic" positions (to use the 
term m Its widest possible sense) and the number of statements 
that can be framed to express any single one arc, or course, 
enormous. Yet the practical situation demanded that the final 
lut be extremely brief, Wo tried to secure brevity in three 
ways. (1) We allowed certain goals between which distinc¬ 
tions are customarily made in philosophy and which y« appear 
to be closely related to one another m the minds of most stu¬ 
dents, t“ stand together m a single statement * (2) Certain 
goals such as the attainment of Nirvana, which are familiar in 
the history of thought but which are not commonly held by 
American college students, were omitted (3) Certain goals 
common in student philosophies but in a subordinate position 
were also omitted' 

Closely connected with the selection of the goals was the 
question of what technique should be used. Since the purpose 
Was to get the student to rank the goals ui order of their im¬ 
portance and acceptability to him, the device of "paired com¬ 
parison" appeared appropriate since this technique facilitates 
accurate and easy ranking of various poanbiliuea. Thus each 
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of the twenty goals is paired once with all the other nineteen 
and the student is asked to choose one goal of each pair As a 
result of this process, the goal for which the student has the 
greatest preference will be selected nineteen tunes (i cvtt y 
time it appears) or will have a “score" of nineteen The stu¬ 
dent's second choice will he selected eighteen times (a score of 
eighteen), being rejected only in the pair where it appears with 
the most acceptable goal. The other goals follow with dimin¬ 
ishing scores until die goal which the student finds loaar accept¬ 
able or rejects moat emphatically is reached In short, die 
student who manages to make his choices with perfect con¬ 
sistency will give to goals "scores" which will rank them m 
order,* 

In interpreting the inventory, one looks first at the goals 
which are ranked at the head of the list (The exact number 
of goals considered depends partly on the nature of the par¬ 
ticular goals concerned, partly on the arrangement of the scoia 
in terms of gaps or tics m the ranking.) These goala at the 
head of the list are the statements which most appeal to the 
student Next the goals at the foot of the list are considered 
since the student's philosophy as indicated by the goals most 
readily accepted is often further defined and clarified by the 
goals which he rejects (or accepts least readily) 1 

The way in which the inventory records a philosophy of life 
can probably beat be indicated by an example This student 
before taking the inventory, wrote a brief essay stating her 
philosophy This essay is reproduced here aqd followed by a 


* For KVtnl raBoni which in too nmtplmled far dnnmloo hope, paftci 
‘cMMbnncy" k rare, and m arc common id im nnkJap n»fe by iludsau 

' In interpreting the Inventory, umilly little weight k gt»«n to ibac pah 
which appear in ihe middle of die Ibl The tnui lor the proctdnra are huh 


which appar I 
pfulmophicil t- 

StatHiinJly, aktrme doruBooi act of cnona milch Im likely ro occur Ixcium 
of pnra chine* 

rfcfemphmUy, tbs bun concept* cl many pcmiram can be mud by intMln 
■datively f.w ol tha atatamta a (tin liar and by rajutioE a few oihm. For a 
particular phsomphieal psahtan, many <rf ihctc gnb luted may be irrdcvani or 
memlogjM. The form of (he uat doa not, hdwaver, pinnit die Kudnc id dkaid 
that nail liceralty He mnu coaruiua to meke chora iavoWng diem He leafe 
to mirk thota goal, to which ha u Indifferent Minewhat lower than ibo«« gmli 
whwh he incfO la ftimitna of ha put| Ron. yet smawhii higher (ban daua rail 
which fall prnilaw rv-ccmrily rejaro. Id other wnrdi, the gnaii toward whfctabe 
■ neutral or odrifemit tend to appear In the middle a hit ranking, 


students' general coals in ufe 93 


luting of the order in which this student subsequently ranked 
the goals of the ipvemoiy. The cose is fairly typical in that, 
while the statements in the philosophy are not verbally identical 
with the wording of the goala in the inventory, the general 
nature and tone of the phdosophy arc reproduced 


Afy Pkilaiofky of Life 

I And it difficult tn scare my philosophy of life One's philosophy 
U far from being on immutable Thing, ana mine is constantly chang¬ 
ing from contact wuh the outside world. The beat I con do is to 
ante the basic pnnaplea which are elementally stable, 

I firat of all think that life ■■ to bo enjoyed Happiness no matter 
how hnef, leaves a lasting impression that Uvea on from one period to 
the next Its memory fills the more somber moments with a hope 
for better thinn to come If 1 am happy I can be successful, and 
rucctss gives the impetus to do better irungs, 

Every man on earth !■ here for some purpose, I believe that ] 
can attain that purpose by doing the best I can with everything 1 do. 
The thing at wblcb I was intended to excel will show itself ut the en¬ 
joyment! derive from doing u At tho present time my greatest 
source of joy is in wnnng I cannot say yet that writing is my talent, 
What I write la lnvnainre and sometimes I become very discouraged 
when my words do not match my thoughts But to know that 1 
have written a line full of life and color gives me more pleasure then 
any Other thing I do 

I aiq oar concurred with the bfa to coma after this ona. I do not 
pretend to know what it holds for me or how I can change it. 1 am 
concerned with the present It u of too much importance to be con¬ 
sidered u l doormat to the house of the next life. The present u a 
housein itself and it must bo lived in. I believe that if this Life were 
twt meant to be enjoyed it would not hold so many sources of joy 
It is not my ambition to be remembered lor generations to come 
because of some mat secompliibnwit If I succeed In doing one 
thing really well, I shall be tadufod. It was never intended chat I 
bs a lauous person, but ] am going to see to H that I am a successful 


"Score" 


{memory Score} 
Coot 


18 Gettuig as many deep and lasting pleasures out of life as lean 
17 Promoting the most deep and lotting pleasure* Tor tho greatest 
number of people 

17 Self-development—becoming a real, genuine perron. 

Fine relations with other perrons 

place for myself in the world, getting ahead 
the specific problems of life u they arise. 

—-L eontaotment, itiUntu of spine, 
x WO people and things. 
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10 Serving the community of which I ant a part 
9 Self-sacrifice for the sake of a better world. 

9 Living for the pleasure of the moment 
7 Serving God, doing God's will 
7 Achieving personal immortality in heaven. 

6 Self-discipline—.overcoming my irrational emaDons and sen¬ 
suous desires 
6 Doing my duty. 

6 Survival, continued existence. 

4 Bang able to "cake it"; brave and uneomplaimng acceptance 
of what circumstances bring 
3 Finding my place in life and accepting it, 

2 Realizing that I cannot change the bad features of the world 
and doing the beat I can for myself and those dear tn mi, 9 
I Security—protecting my way of life against advene changes 

In her essay, this girl Ant mentions, as an aim in fife, the 
desire to be happy Thu denre li reflected in the goaf which 
heads her list in the inventory, VGettmg as many deep and 
lasting pleasures out of life as I can " (Of die statements in 
the inventory this is the expression nearest to "happiness") 
The rank she gives to “Peace of mind" indicates the same atti¬ 
tude The eociul aspect of this same aim appears in two other 
goala she ranks high. “Promoting the most deep and lasting 
pleasure! far the greatest number of people" and "Fine rela¬ 
tions with other persons ” 

The second aim emphasised in the eeaay is the desire to be 
successful, to do the best she can in everything In the inven¬ 
tory this aim is indicated by the high ranking of the goals of 
“Self-development" and “Making a place for one’e self in thfe 
world " 

In hor essay, this student stresses the present and Is specifi¬ 
cally indifferent only to the problem of “the life to come after 
this one." Hence the goal of "Handling the specific problems 
of life as they arise" ranks high tn her list and that of "pemnal 
immortality" fall* in the lower middle 

At the end of her list come, as might he expected In the coxa 
of one who wishes fa make a place for herself in the world, the 
more passive goals of "Finding my-place in life and cheerfully 
accepting it" and merely “Being able to take it 
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Possibly these brief comments will suffice to show the more 
important relations between the essay and the inventory* 
though practice gained through interpreting a number of scores 
makes the results more meaningful to the user Some differ¬ 
ences between essay and inventory may also be noted For ex¬ 
ample, the interest in writing, stressed in the essay, does not of 
course appear in this inventoiy On the other hand, the inven¬ 
tory reveals the student’s point of view on a number of issues 
which she did not mention in her essay Unfortunately a sin¬ 
gle example cannot illustrate the many varieties of viewpoint 
and the striking contrasts between them which can be indicated 
by the inventory, nor can it show the analyses and comparisons 
possible for the scores of groups of students 1 

It u still too early to make a final evaluation of the inven¬ 
tory Data are snll coming in, and studies of the validity and 
other aspects of the instrument are still being conducted by the 
staff and by the cooperating institutions, Nonetheless it is 
fair to say on the bans of the evidence now avadable that the 
inventoiy has proved extremely useful and seems a reliable in¬ 
strument for the purpose for which it was intended 

* ol uml «ni*p* ot lOuiati ini idulu an ba found m Suf Nrmi 
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cost was enormous, but Americans have always paid hand¬ 
somely for their entertainment. In the field of business and 
industry, competitive advantage, often cut-throat in nature, 
took, precedence over other considerations Extended to per¬ 
sons, the principle found expression in hiring, firing, and fierce 
competition for jobs 

Even in education the pressures against reasonable selection 
were legion, and there had been marked lag in the adjustment 
of opportunity to individual and social needs For example, 
before the College Entrance Examination Board was founded 
m 1900 the colleges had been "evaluating" secondary schooling 
by means of records not even superficially comparable. When 
they had muddled through to elements of a rational policy for 
admission, some presidents still objected One, more naive or 
pragmatic than the others, "pointed ouc that a college might 
wish to show special favor to sons of large benefactors, to sons 
of trustees, or to sons of public men of importance who pre¬ 
sumably would have difficulty in meeting the announced ad¬ 
missions requirements " 1 

Eventually a system of personnel selection may develop 
that can with justice be called scientific Those who have 
studied the problem know that on theoretical grounds an in¬ 
finite number of such systems exist as possibilities, Two 
questions would have to be answered if a particular system were 
chosen for practical application’ Which of the possible systems 
would be mart uitjvlt, and Who would select itf According to 
democratic ideology, the citizens would decide and choose. It 
would be useful to the extent that its results were sensed as- 
beneficial to the entire, society, it would be selected by the 
entire society. If the urge for scientific selection were com¬ 
pelling, we would now be searching for our undefined or basic 
terms; we would be exploring the postulates smallest in number 
but tuefvl to attaining our aims. With these goals reached, our 
conclusions and their application would follow as inevitably as 
night follows day But generations may come and go before 
public opinion can forge a method of selection worthy to be 
called scientific. 

rc* * wi ‘ ** 


MAJOR STRATEGY VERSUS MINOR TACTICS IN 
MERIT ADMINISTRATION 

FRED 5 BEERS and CECIL R. BROLYER 
Social Security Board 

Civil service or ment systems are relatively new to the 
American scene of govemmsne Even the most venerable of 
them are only two generations old, but between 1935 and 1944 
the number skyrocketed until there were state-wide systems or 
ones that included a combination of departments in all but 
three states Stimulation for this phenomenal growth came in 
part from federal legislation, particularly the Social Security 
Act, which in 1939 was amended so that selection of personnel 
on q ment basis was one of the requirements for receiving grant 
in public assistance and unemployment insurance 

Passing laws is risky bustnoss—witness the prohibitioi 
amendment. Civil-s&rvice legislation haa sometimes been 
passed in nearly as precipitoue a fashion Although the re¬ 
sults have been notably better, they have left little doubt that 
statutes alone, indispensable as they are, cannot make a merit 
system Practice must keep step with legal precept if public 
acceptance is to be sufficiently durable to make laws effective. 
Philosophically inclined members of the legal profession say 
that 80% of the people must be doing voluntarily what a law 
dictates if the 20% who need coercion are to be brought into 
line. 

For more than a century the Amencan tradition of per¬ 
sonnel selection was opportunistic. Officeholders wore dis¬ 
possessed regularly with each change in administration, Hav¬ 
ing an average expectation of tenure amounting to two years, 
they spent the fint year learning what they were supposed to do 
and the second helping to re-elect the chief administrator 
This practice was a main source of popular merriment. The 
07 
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Meanwhile we must consider available alternatives Among 
these the merit system seems best, although its imperfect tech¬ 
niques can be much improved as this article will attempt to 
point out 

In their present stage of development merit systems cannot 
pretend to science At best they represent a compromise be¬ 
tween the rational and the instinctive, drawihg as they can on 
elements of the scientific that lie snugly adjacent. Internally 
their promise of development lies in a frank recognition that 
theira is an art; that, as in clinical median^ their art u un¬ 
worthy of the name if any poaaible conjunction with science is 
overlooked or set aside Externally their hope lies in shaping 
public opinion by the unswerving integrity of their practice and 
the truth and cogency of their precepts The basic principles 
should be and are self-evident They should both reflect and 
illuminate the democratic ideal. 

Five basic principles for a ment system of public adminis¬ 
tration may he said to have reached the stage of general accep¬ 
tance In a democracy they are axiom sue (1) open competi¬ 
tion as the method of choosing public employees, (2) selection 
by practical and scientific methods or the most competent per¬ 
sonnel available from open competition, (3) equal pay for equal 
work, (4) a career service conditioned upon meritorious per¬ 
formance of work, and (5) the right of appeal from any or all 
personnel actions 

It is unfortunate but not, leu than human that disagree¬ 
ments over the minutiae of carrying out a principle sometimes 
become so noisily vociferous that the still, small voice of the 
principle ia lost in the general clamor. Mere contentiousness, 
however, is a minor ill. More critical distractions from the 
principles arise from legitimate differences of opinion and the 
swift crosscurrents of deadlines to meet, techniques to refine, 
inefficiencies to scotch, rituals to perform, red tape to unwind, 
irato applicants to soothe, and special privilege seekers to 
confront. 

In the hurly-burly it is sometimes forgotten that ment sys¬ 
tems are never ends in themselves. They oust to serve the 
operating agencies Rut merit systems should be given ad- 
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minUtrative strength equal to chat of the operating programs 
and should have their unqualified support Lacking thie, they 
may in tune degenerate into irresponsible and malodorous 
adjuncts of the spoilt system. The relationship hetween the 
operating agency and the merit system is reciprocal As the 
one proepen, so does the other. If the one sinks into respect¬ 
able somnambulism, the other is desolate also. 

Some of the complexities involved in applying basic merit 
principles will be apparent if we examine each principle sepa¬ 
rately (1) The principle of o fun competition as a generaliza¬ 
tion is especially allunng in a democracy. When II L Men¬ 
cken mins salad days translated the Declaration of Independ¬ 
ence from sonorous Johnsonese into American, he voiced the 
underlying principle in language dear to the heart—"Every¬ 
body is as good as avniybody else and maybe a damn sight 
better.'’ 

In greater or lesser contradiction of the principle, minimum 
qualifications of education or experience or both are often act up 
as obstacles to competition The rational Izacion is that these 
hurdles constitute protection against the cose and waste of fail¬ 
ing large numbers of applicants, experience having shown that 
most people with 4th grade education and wheelbarrow work- 
hutoiy flunk out on examinations for white-collar jobs Per¬ 
sons of great natural endowment are, of course, leas dependent 
upon formal education than others, Americans like to think, 
however, that Lincoln, the Wnght brothers, and Edison are 
typical rather than exceptional. So-called minimum qualifi¬ 
cations may aomerimes be subordinated to the interests of 
special vocational or professional groups; rf severe restrictions 
are imposed, they may become subversive of the principle of 
competition they were designed to facilitate. The danger is 
accentuated by the fact that amount rather than quality of 
experience is credited (Comparable data on quality of experi¬ 
ence are almost unattainable.) 

Local residence is almost completely indefensible as a hurdle 
to competition Yet it is widely accepted, perhaps because it 
expresses the yearning of the community to regard itself and its 
members as better than . By way of illustration it is rumored 




that every faithful citizen or Boston hopes some day tn mov 
New York and not to like itl “ 

Preference for special groups limits the application of 0 p«i 
competition. Lady Bountiful scattering sunshine by her pre *. 
ence and gifta from her well-stocked larder is a national Jdol, 
Except in poetry we seldom think chat “Earth gets its price for 
what Earth gives us" For example, recent polls show exten¬ 
sive public support for continuing the tradition, of preference 
in the public service for veterans, their wives,' and kin. What 
would be tho offect on public psychology if proposed legislation 
read about as follows If you are not a veteran, you will have 10 
points subtracted from your competitive score on whatever 
examination you take for a position ui the public service, rf yo)1 
are a veteran but not wounded, you will have 5 points sub¬ 
tracted. It u not only in our efforts to befriend veterans that 
we fall into the illogical quicksand of something for nothing. 
Wo do so elsewhere. All of us are appalled at the “mounting 
death rato owing to heart disease.” We are brought up short, 
however, when we are aaked what death rate we should like to 
see rise if that for heart disease could be reduced 

(2) Selection of the best available as a result of competition 
commends itself to the rationally minded. Most commonly 
the process consists of a written examination, a rating of educa¬ 
tion and experience, and an oral interview. Resulting scores 
are combined by differential weighting, each weight presumably 
determined from the relative appropriateness of the pare scores 
to the requirements of the job. At the nak of over-simplifica¬ 
tion. we may regard the written examination as a check on the 
powers of thinking and the extent of appropriate knowledge, 
the rating of experience, a check on stabihty—the staying pow- 
era of tho candidate; and the oral Interview as a check On hb 
histrionic abilities—how good he is as an actor. 

It is in the area of the wntten examination that personnel 
administration today cornea most closely into conjunction with 
science Full advantage should be taken, but often Is not, of 
this fact Scientific measurement has come about within the 
memory of men stdl living, Between the early attempts of 
Gallon and Cat tell to apply to human traits measures analogous 
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to those used in physical science and the appearance approxi¬ 
mately two decade* later of the Army Alpha Ilea a period of 
significant advances. Since that time, in school#, colleges, and 
in our armed forces this science has come of age, both In theory 
and in application. It has been expanded to include special 
skills and aptitudes, attitudes and interests, and is being ex¬ 
tended to include relatively unique differential traits In merit 
administration the advantages this science offers in “arranging 
candidates in order* 1 is recognized 

Rating education and experience has been an armchair exer¬ 
cise, About all that (s reliably measured is amount of time 
spent in school or at work; the quality of the experience and its 
effect upon the person are ignored True, those who have gone 
to school for a long time or have held responsible positions are, 
if a random group, more likely to succeed in similar circum¬ 
stances than a corresponding group of the same age who have 
not. But extensive studies by competent observers have 
shown, for example, thit there u a college for every level of 
ability in almost every State, and that variability of achieve¬ 
ment within colleges is marked. But neither of these facta is 
considered in evaluating education Although the studies of 
work experience have not been so extensive nor perhaps so care¬ 
fully made, little strain is put on the imagination in recogniz¬ 
ing that evaluating work-experience and schooling are equally 
comply. What is true of the one would more than likely be 
true of the other. 

wo much reliance can easily he placed upon work- 
experidiee {a made clear by three commonly practiced adminis¬ 
trative devices plus a factor that we shall call "survival" 
When an administrator has on his staff a difficult or inept per- 
•on—but not so bad that dismissal it, under the rules of tenure, 
relatively simple—be watches his opportunity to “unload” that 
employee on someone else. The inept one moves on to his new 
position happily unaware of what has happened to him. And 
he may bo similarly Unloaded several time* carrying with him 
as he moves crab wise a record of lengthening and satisfactory 
j experience. If he were given an unsatisfactory record, he could 
qot be uploaded. Th< second administrative practice is famili- 
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arly known os “lucking upstairs." For equally good, or bsd,‘ 
reasons the unwanted individual is eliminated through promo¬ 
tion or reassignment within the organization but at a higher 
level. The third practice it "holding down " That is, an em¬ 
ployee may be so valuable in a low position that the administra¬ 
tion cannot affbrd to advance him In any event the record 
of the worker, as seen by an objective observer, ™«bi at 
a parr of the truth. 

At for the factor of “survival," it it among the commonest 
of influences contributing to unreliable rating of work-experi¬ 
ence. In every orginixaticjri people tend to get bored. Those 
moat ambitious, energetic, and competent will find opportu¬ 
nities for advancement more readily than others. If top mao- 
age men t by too little or poor leaderahip contributes to this 
centrifugal force, the better people rather than the weaker ones 
will tend to find elsewhere jobs more suitable to their talents. 
These vacated jobs are likely to be filled from among those who 
are loft. Eventually persons having long; solid, C+ experience 
predominate. This is the factor of survival that over a penod 
of yean contributes to high ratings on experience for mediocre 
individuals and, incidentally, may ataff die top jobs with in¬ 
competents. 

Oral interviews could be a very useful part of the selection 
process Although they are extensively used for positions re¬ 
quiting contacts with other workers and with the public, their 
value is sometimes questionable. Pan of the difficulty results 
from failure of examiners to recognize the limitations pf ;he 
interview and to refine techniques that will point up its<virtuea. 
Essentially the interview should be a check op the candidates 
pwere of seeing. That everyone is an actor will be imme- 
diately apparent if we reflect For a moment on what happens 
dunng every conversation We keep the lid on those things 
that would give offense and try to express only those that will 
please or that will accomplish our ends with the least amount 
of friction ( When the “oral" is used as the casting dimtot 
employs the tryout, we shall have begun to refine it for mojO 
effective use. -The easting director ask a the people who conw 
before him to ‘register" this or"register" that. His applicants. 
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who aim to please, respond accordingly} and the director then 
makes a selection from among them according to the require¬ 
ments of the part that ii to he played “The part” and "the 
requirement* of the part."—these are the terms that need 
definition in any development of devices to arrange competitors 
in relative order of skill 

To sum up, selection normally consists first of a written test. 
Such a test can and should be carefully prepared since a science 
for doing this is available. It is the best method of determining 
the would-be public servant’s knowledge and his ability to use 
that knowledge The rating of training and experience is a 
second factor, contributing to the selection process Has the 
applicant been through the conventions of society, both educa¬ 
tional and social, that are thought to be prerequisites to com¬ 
petent performance on the job? Sole reliance on these con¬ 
ventions, prerequisites, or minimum requirements implies a 
faith in two other propositions patently false, namely, that all 
individuals have equal endowment and that all respond equally 
to experience. If these last two propositions are not true, then 
there is always a real probability that lomeone can bo found 
who could do a competent job although lacking minimum 
qualifications Nevertheless, the establishment of some con¬ 
ventions u administratively desirable and even essential But 
their promulgation by fiat should be made in a state of self- 
awareness rather than in a state or stir-delusion These con¬ 
ventions simply apply two questions to the competitor la he 
reasonably well equipped for the position for which he is com¬ 
peting? Does Ins expenmee indicate dependability? Because 
of limitations m evaluation, in tbe individual, and m society, 
such ratings are purely judgmental today, that is, they are still 
unscientific Finally, the oral interview can and does con¬ 
tribute to the selection process. Its purpose needs clarifying 
and it* techniques refining The three in optimal combination 
yield better results than any one alone 

(3) Equal pay for equal work is a necessary complement to 
open competition and selection of the beat qualified if a merit 
system is to be productive of an effective working spirit. Few 
forcca are more disruptive of staff morale than inequities in pay 
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and rank. Prom the principle of just recognition for work done 
stems the durable structure of an oiganization, tbe position 
classification and pay plan In principle equality and fairness 
are axiomatic, They need no defense. But they are as diffi¬ 
cult and elusive to apply as they are ielf-evident There ia as 
yet no science of organization, no clear dependence of pay 
eeales on ineluctable fact Even more distressing is the ap¬ 
parent ease with which known elements of human psychology 
and d os e-to~ earth experience can be ignored. 

If keen observation and fidelity to meaning of facta ean be 
called psychology, the novelists are often to be classed among 
the beat psychologists As Charlotte Bronte wrote in the 
"Editor’s Preface to the New Edition of Wuthering Heights”' 

The writer who possess^ the creative gift awns something of 
which he is not always majter—something that, at rimes, strangely 
willi and works for itnir He may lay down rules and devise pnno- 
ples, and to rules and principles it will perhaps for yean lie in sub¬ 
jection, and then^ haply without any warning of revolt, than comes 
» anw when it will no longer consent 10 "harrow the valleys, or be 
bound with a band In the furrow" When, refusing absolutely to 
moke ropes out or sea-sand^ any longer, it seta to work on stamp, 
hewing. . Be the work grim or glorious, dread or divine, you have 
little choice left but qumcent adoption. As for you—the nominal 
artist—your share in it has been to work passively under dictates you 
neither delivered nor could question. . , If the result be attractiva, 
tjui World will praise you, who little deserve praise, if It be repulsive, 
the same World mil blame you, who a brunt ai little deserve blame. 

On firat reading one gets the impression that here is, per¬ 
haps, the best definition of genius he has seen But closer re¬ 
flection, stimulated by >the shift in point of view from the 
impersonal “he" to the very personal "you," suggests that 
Charlotte's description is not of genius at all but of everyone 
who ia human The ordinary or garden variety of classification 
and pay plan under a merit system leaves much undone that 
might with a little effort mold auch plana closer to the heart’s 
desire. The typical pyramidal concept leaves out of account 
much of the universal human urge for elbow room—freer play 
for talent; At the top administration, next supervision, then 
prof canons and specialized vocations, and finally the workers. 
Even at the top it ia quite possible to search dozens of plana 
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and nor find one that would make provisions elastic enough to 
attract a man like William James—physician, psychologist, 
philosopher, man of letters; or Benjamin Franklin—artisan, 
scientist, inventor, diplomat, statesman 

Inelasticity, however, Is not the only weakness Of greater 
jeopardy to the principle of fairness, but perhaps more easily 
remedied, is the failure of many operating administrators to see 
lha manifold advantages a classification and pay plan can con¬ 
tribute to good administration. Too often they regard the 
structure as a hampering rather than a facilitating device; by 
circumventing the requirement* they put themselves jn the 
position of the sick man who bruahes aside his physician and 
clings to hw amulet Merit systems must contribute more 
than they da now to informing and instructing the operating 
agencies—administrators and workers alike, 

(4) Fourth among the major principles of mem administra¬ 
tion^ the requirement that the system provide for a career 
service based ort merit. Basic to this principle is tenure of 
office for those who have been selected as the best available 
knd who have qualified for permanent appointment by having 
served successfully a working-teat or probationary period 
Once they have passed muster it ia in the interest of the public 
service that they be given reasonable assurance of holding their 
job a. With this assurance they are free from fear, and can 
direct their energies to increased job efficiency. Bernard 
Shaw, however, cautions rhoie who start at the bottom and, 
hopefully, climb the proverbial ladder rung by rung. You 
don’t learn even to hold your own, he says, “by standing on 
guart but by attacking and getting well hammered yourself ” 
A policy of projaouon-from-withu) » another dement of a 
jareer service. It is earned to extremes when It is so nearly 
like the human drculatory system that elaborate preparations 
and intravenous injections are necessary for the purpose of in¬ 
troducing a little new blood. Very often new blood brings new 
lue, as many a soldier can testify “Permanent tenure” is not 
an Unmixcd blessing It has even been called a necewary evil. 
Those who accept it rb a principle of administration and those 
who profit from it as employees ahpuld understand its hazards 
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And the limitations on its effectiveness As part of a career 
service, some method of evaluating th? worth of those with the 
prospect of promotion is necessary Recourse here to the basic 
principles of selection is indicated—written promotional ex¬ 
aminations with a broad base of competition, plus service or 
efficiency rating as partial determiners of "tho best available." 

Service ratings leave much to be desired. They have inhw- 
ent limitations. They put a premium on conventionality and 
often on the mediocrity that never swerve* from the beaten 
path They do not foatcr understanding 0 f the brilliant em¬ 
ployee whose contribution this week it more than noteworthy 
but whose unexpected absence the Mowing week leaves the ad¬ 
ministrator gnashing his teeth Service ratings do, however 
bring out into the relative open, so that impartial observers can 
we them, the supervisory judgments that would otherwise be 
made behind doted dooia and acted upon arbitrarily in star 
chambers Moreover, the rating process can be refined If 
employees share in it and understand its advantages as well as 
its limitation* it can serve an exceedingly useful though limited 
function 


— ' ' T-* , ,,T ” is me ngat of appeal. 

•nils right rests on the theoiy that the mdmd&al under merit 
administration has basic rights that cannot bp ignored and that 
rile administrator has responsibilities that he must discharge 
Appeals usually lie when an applicant has been refused the 
privilege of standing an examination; or if he reels that he has 
been unfair^ rated on one or more parts of the selection 
process, or rf he challenge* the fact that he was not certified 
ot that his name was improperly deleted from a register Ap¬ 
peals frequently lie for layoff, demotion, suspension, and almost 
always for summary dumml 

Many defensible differences of opinion exist over particular* 
W ix eiy ° f "PP* 1 * 1 ' actions covered, the nature and 
calibre of rile appeals body and the manner of appointment, the 
quaition of informing the woricer of his rights, and the question 
or the administrator's responsibility once an appeal has resulted 
in a finding But there u little if any disagreement among 
merit or cm! service systems that the right of appeal is cardinal. 
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In this article merit systems have been criticized—not chat 
they may be destroyed but that they may be improved The 
only «Itermtive to selection by merit u selection by personal 
whim In that direction lies chaos Thus, society is left with 
the task of making its "merit system" more workable, more 
reliable, more internally consistent—in short, more nearly 
scientific. Enough experience with merit systems is now avail¬ 
able that some of the malfunctioning are apparent Knowl¬ 
edge is avadable that could be used to correct these malfunc¬ 
tioning^ Improvement of merit systems is contingent upon 
their scientific development, their acceptance is dependent 
upon a greater public awareness of what ment systems are, 
their perfection waits upon a demand that they be what they 
could be 

In Tfus Faith That Heal:, Sir William Osier, noblest physi¬ 
cian of his day, says, "Nothing in life is more wonderful than 
faith—the one great moving force which we can neither weigh 
in the balance nor test in the crucible " Through its Faith in 
democracy this nation is philosophically committed to the 
merit principle Imperfections are transitory. The “sub¬ 
stance of things hoped for" is scientific selection of the beat 
qualified public servants 
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The chief function of a teacher is that of directing and moti¬ 
vating pupils toward the attainment of desirable educational 
objectives. In the performance of this function testing con 
play a very important part When objectives are adequately 
defined and teBta devised which are valid with respect to them, 
the extent to which objectives are being attained can be mea¬ 
sured. Furthermore, such tests define the objectives for the 
pupils and motivate the pupils toward them When the results 
of such tests are adequately analyzed and interpreted, the 
teacher obtains a means of better orienting instruction and the 
pupils secure motivation through knowledge of progress 

Construction of exercises and analysis of the data resulting 
from their use ean make objectives more definite and more 
meaningful to the teacher foe creation of novel exercises 
and the analysis of data pertaining to them may widen the 
scope of objectives recognized and ultimately realized in in¬ 
struction 

Instructional objectives are moat usefully defined in terms 
of observable behavior. Each specific objective should be an 
answer to the question "What should the pupils be able to do 
as a result of ins miction?" Instruction which produces the 
abilities to do certain things, should concomitantly develop the 
attitudes, interests, and ideals which motivate their doing. 
Instead of the general and intangible objectives “good citizen¬ 
ship,” "appreciation of good literature,” and "scientific 
method,” specific objectives formulated In terms of observable 
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behavior may include, “Presenting arguments in support of the 
elimination of the general property tax based on factual evi¬ 
dence critically analyzed and evaluated," ''selecting a short 
story for leisure reading on the basis of the following cri¬ 
teria. -"rejecting a conclusion which goes beyond the data " 

Many instructional objectives must be concerned with fac¬ 
tual information, or knowledge In the selection of facta con¬ 
sideration should always be given to the contribution such 
knowledge cm make tp the types of behavior illustrated in the 
preceding paragraph The thinking necessary for adequate 
performance of activities recognized as the really worth-while 
objectives of instruction must be based on knowledge. Infor¬ 
mation which makes no recognizable contribution to such 
thinking, or to the further learning which may contribute to 
such thinking, is not worth teaching. It is also not worth 
testing 

While teachers usually contend that their objectives are 
not restricted to the memorization of miscellaneous! unrelated, 
and often trivial information, the tests used by most teachers 
are convincing evidence that their actual objectives are thus 
restricted When objective tests are made by teaehere the 
exercises are most often of the true-false or multiple-answer 
types am) the content qf there exercises is wholly based upon 
information given in the text When essay testa arc: con¬ 
structed and used, the questions are factual in character and 
are scored only for the facti remembered. 

Tests should be designed which measure knowledge of facts 
True-false or multiple-answer exercise* can be efficient means 
of measuring such knowledge. The possibility of represen¬ 
tative sampling of pupil knowledge is one of the advantages of 
objective testing In writing such exercises the teacher should 
be critical in the selection of the facts to be tested One useful 
criterion in the selection of facts is *he degree of relevance of the 
fact w some important general concept nr principle which can 
be applied ip the solution of some problem involving reflective 
thought. Consideration should be given to whether or not 
klXswjedge of the specific fact contributed to further learning 
In many fields progress in leartung is contingent upon the syn- 
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thesis in the mind of the pupil of an ever growing body of 
factual knowledge In dunking about an important con¬ 
temporary social problem tho pupil may require a knowledge 
of numerous historical facts relevant to the trend which bu 
created the problem Senes of factual objective exercises may 
be useful in determining tho extent to which such knowledge 
has been attained, 

In writing true-false exereiies certain precautions should be 
observed Broad generalizations should usually be avoided, 
Such worda at "always,” "never,” “none,” "only," "all,” and 
"every" are obvious dues to the falsity of certain statements 
when the pupil can readily think of exceptions On the other 
hand, the statement "all echino derma live m salt water” rep¬ 
resents a difficult true-false item to one whose biological knowl¬ 
edge is not extensive. In multiple-answer exercises the incor¬ 
rect completions should not be too obviously incorrect. Eacb 
completion should be plausible It is frequently effective to 
write multiple-answer exorcises in which a "best" rather than 
a "correct” answer la called for. Such exercise* tend to accom¬ 
plish more than the measurement of memorized information. 
All, or moat, of the completion* may be "correct”; the pupil 
must judge which completion is the "best,” In a recent social 
acienee comprehensive examination the directions for one senes 
of multiple-answer exerases asked the student to identify the 
alternative suggesting the "most significant relationship” be¬ 
tween the two things mentioned. The fine exercise in thu 
senes is given below 

1, Minor party—social and economic reform • (A. Minor partus 
are usually characterized by radical platforms; B minor parries sel¬ 
dom win an election, C reforms agitated by minor parties are soraa- 
tlmq adopted and enacted into law by the major partus, D unsuc¬ 
cessful minor parties ultimately pus out of existence, E, the major 
artus have the advantage in organization, fundi, and prestige and, 
ante, are more successful in promoting reforms ) 

While most of the answers are "correct," alternative ,f C 
represents the most significant alternative In a recent biolog¬ 
ical science comprehensive examination each exercise began 
with a statement to be explained. More than one of the alter¬ 
natives were "correct,” but only one was accepted as the 
"explanation.” The following exercise is an examplei 
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Slat muni In embryological development certain itructuret de¬ 
velop id a manner analogous to the conversion of a passenger boat 
mto an aircraft earner (A The monila changes Into a bhatuh 
which change! into a gastrula, D part of the placenta 11 formed from 
the wall of the uterui of the mother, C the mesoderm gives rue to 
the hearr, blood vessels, blood cells, lymphatics, kidneys, and certain 
other organa and structures, D in the higher vertebrate*, the middle 
ear cavity, the eusiachian tube, the thymus gland, and the parathy¬ 
roid gland* develop from gill slits nr arches, E the ectoderm gtvei 
me (o the epidermis and ro the fining of the mouth and anal aper¬ 
ture* ) 

The answer "D" is the alternative which best explains the 
statement. This example illustrates another very important 
matter If the teacher desires to measure how well pupils can 
handle questions involving thought rather than memory, an 
exercise must constitute a novel problem. It is improbable 
that the students were taught the particular analogous relation, 
■hip implied in the above exercise. If they were taught this 
analogy, the crerciae will measure only the extent to which the 
analogy ia remembered If, however, they were merely taught 
the facta represented by alternative "D n then the relating of 
this alternative to the introductory statement becomes an act 
til thought transcending the utilization of memory. The ex¬ 
amples just given alio illustrate the fact that exercises which 
are simple in form can be effective m the measurement of be¬ 
havior involving thinking. It is not necessary to construct 
exercises involving complex directions in order to obtain such 
measurement, "Hie important thing is that the content of the 
exercise should be to some extent novel to the pupds 

Ducnmtnaiwe Thinking 

Onq means of securing the novelty referred to in the pre¬ 
ceding paragraph is to bring together in a senes of objective 
items numerous facts whose classification in certain ways will 
involve the ability of the pupil to do discriminative thinking 
and to synthesize his knowledge. Let us suppose that pupila 
in physics have hcen taught certain facts pertaining to sound 
and, at a later date, certain facts pertaining to light Let us 
suppose further chat the teacher has not stressed the similarities 
and differences of sound and light phenomena The good 


teacher would probgbly stress these things, but For the sake 
of our illustration, let us suppose that this is to bo postponed 
until the test hu been given If this has been the case then 
die following senes of exercises should involve discriminative 
thinking and, as this thinking u taking place, require a synthe¬ 
sizing of knowledge by the pupil' 

On the line preceding each of tho following items, write the letter 
A if the item is true of sound 
7 ff the Item is true of bght 
C if the item Is true nf both 
_ Its velocity in water is greater than m air. 

_ It can be reflected. 

_It can travel through a vacuum. 

_It can be refracted 

_ Etc. 

A number of examples of this very useful general type of 
exercise are given below from various fields In each case only 
a Tew of the items are listed 

In each situation below, an individual or a group of individuals » 
seeking protection or am stance On the line preceding each of the 
following items, write the latter 
A if the Federal government is responsible 
7 if the itete government is responsible 
C if both governments are responsible 
D if neither government is responsible 

_The New York Life Insurance Company wishes to open up 

agencies and tell insurance in Oregon, 

_ Mr. Jones receives in payment f1,000 In bills which he presently 
leirns are counterfeit 

— A Chicago visitor from Fort Wayne. Indiana, suffare savere in¬ 
juries when hie car is wrecked because of defective pavement on 
76th Street , 

M Convicts escaped from Joliet Penitentiary arrive m Des Moines, 
Iowa, hold up a bank, and are seized and held by the local au¬ 
thorities. 

_ Etc 

On the hne preceding each of the following Items, wntn the letter 
A if the sentence is fragmentary 
7 if the sentence contains a comma fault 
C if the sentence eonteina e dangling modifier 
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D if the sentence exemplifies lack or parallelism or faulty parallelism 
7 if the sentence u correct 

- One day I feel ae though 1 could lick the world, the next day I 
feel like a swatted fly 

— - Upon returning from the store, my homework requires my 

attention. 

— As we walked along the hall, where a large photograph of Roose¬ 
velt hung. 

_ All of the boys bang gone, most of the manual labor was dona by 
the girls 

— Bemoaning her stupid lot, the army and navy claimed all of her 
friends 

— A freshman learns to study with regularity, to play with enthusi¬ 
asm, and co-operation 

— Etc. 


On the line preceding each of the following Items, Write the letter 
A if the statement is true of the Lyadas 
B if the statement is true of rhe Adorns 
C if the statement is true of both works 
D if the statement is true of neither work 

— The poem was wntren ui memory of a dead fnend 

_ In loan as well aa content the poem displays a deep personal grief 

— The poet ornaments his verse with many classical all linens. 

— Etc, 

The following type of exercise can be used in a variety of 
school subjects and is effective in measuring how students can 
make comparisons.* 

On the hne preceding each of the fallowing paired items write the 
letter 


A if the item at the left of the page is of greater magnitude than the 
item at the right 

£ if the item at the right of the page is of greater magnitude than the 

”' item at the left 

C If (he two items are of equal magnitude 

-- Amount of glucose in blood,. Amount of glucose in the 

entering the liver 4 hours after blood leaving the liver 4 hours 

• mcil u eaten after a meaTfe eaten. 


-™ Amount of absorption of foods. 
by stomach. 

—■ P«wnca*e of urea in blood, 
entering the liver 


. Amount of absorption of foods 
by intestine. 

.Percentage or urea in blood 
leaving the liver, 
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— Amount of heat produced in. . Amount of heat retained In 

the body, the body ' 

— Etc. 

The samo form needs only alight adaptation to be useful in 
writing chronology exercises in histoiy 

On the bne preceding each of the following paired items write the 
letter 

A if rhe event in Column I occurred before the event in Column II 
B d the event in Column II occurred before t^e event In Column I 
C If the events occurred tt approximately the seme time (within 
about a year of each other) 

Column I Column IT 

— Clayton Antitrust Act ....... . Sherman Antitrust Act 

— Alabama Claims Case .. .. Venezuelan Arbitration 

— Dred Bcott Decision . , . Fugitive Slave Act 

- "Wigwam Convention" .Fon Sumter fired upon 

- Ere. 

It should be mentioned that there should be gome relation¬ 
ship between the paired events significant enough to warrant 
their being paired Care should be exercised when writing the 
H C“ items to give events that occurred simultaneously, or very 
nearly simultaneously Note the qualifying remark with re¬ 
spect to category "C" in the directions stated above 

Exercises of similar form are useful in measuring the way in 
which pupils handle correlated or cause and effect relationships 
In writing such Items where the relationship Is definitely cause 
and effect, the cause should be given first 

On rhe bne preceding each of the folio-/ing paired items write 
the letter 

A if increase in one of the things referred to is usually accompanied' 
by increase m the other 

B If increase in one of the things referred to is usually accompanied 
by decrease in the other 

C if one of the things referred to tends to remain the same when the 
other Increase* or decreases 

— Amount of carbonates dissolved in the water of a river Number 
of dams in the nver. 

— Temperature of dip environment of a bird or mammaL Body 
temperature of the bird or mammaL 
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_ Amount of dissolved salt (sodium chloride) In » ^ven body of 
water Number ol amphibia m *h« water 

_Extent to which men make changes m an area kite at which 

the ana tend* toward balanced equilibrium 
— Etc. 

On the line preceding each of the following paired Hems, write the 
letter 

A if ngnificsnt uicrerae In one of the thing* mentioned hai usually 
been accompanied by significant inereue in the other 
B if ngmReant increase in one of the thingr mentioned hai usually 
been accompanied by significant decrease In the other 
C if one of the thing* does not tend to change significantly when 
ngnificsnt change take* place in the other 
Dollar diplomacy Confidence in the United States oa the pair 
of the South American Nations, 

_Effort* to eliminate the poll tax. Orttory an state ngbta by 

Southern senator* 

_ Effora co liberal ire the Supreme Court, Expression* of con- 
■cmaim with re*pert in the iuus on the part of the general 

public. 

Acceptance of the doctnne of check* and balance* Lobbying. 

_ Etc 

Intelligent Reeding 

The ability to read intelligently in a Geld is an important 
general objective and more efforts should be made to measure 
how well pupils can perform this activity This ability is a 
very important factor in Anther learning Within a field and in 
dealing with practical problems when formal school learning 
ha* terminated. The following exercise illustrates one means 
of measuring such ability. In this example, the material to be 
read is a graph. Such exercises need not be so restricted 
One may use paragraphs drawn from texts other than those 
studied by the pupils and even from advanced texts if the 
teacher wishes to challenge the pupils. When paragraphs are 
selected they should be self-con rained in that the topic is 
treated fairly completely. It is frequently effective to present 
paragraphs which give scientific data and some of the state-, 
menu luted may be inferences which may justifiably be derived 
from the data. Others of the statements may be only partially 
justified or may ho irrelevant. Some of the items of the fa&~ 
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lowing exorcise represent predictions which may not be Jmd. 
lied* 



B. if die Information given ui the chart is sufficient only to indicate 
that the statement Is probably true 
C if the information given in the chart is Sufficient for a judgment 
that the statement » definitely felon 
D if die information given in the chart IS Sufficient for a judgment 
that the statement is probably falsa 
E if the information given id the chart u not sufficient to Indicate 
any degree of truth or falsity in the erateuient 

- Leu money wo* spent in 1930 than in 1929 for welfare and educa¬ 
tion. 


— In 1931 and 1932 the raucnditur* of money for highway purpose* 
was evidently considered a means of combatting the UeprwnocL 

- J n 1940 a much greater proportion of the total expenditures Was 
for welfare than in 1942 



— The increasing amount of money epent by the State for all our- 
pwc* between 1929 and 1940 mu*t have come largely from tuts 
or from Federal granum-aid rather than (ram borrowing. 

_ Etc. 
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Such exflirtcea can be scared in the usual way to obtain a 
total score which show* how well the students agree with the 
key set by the teacher, or the "expert thinker* fn the field. 
Such a total score may answer the question* “To what extent 
doea the pupil read data accurately?" and “How well does the 
student recogmre valid general (ration* drawn from data?” It 
is possible, however, to obtain other icons which reveal certain 
characteristic* of pupil thinking. For example, a count of the 
number of M B" items marked "A," and “D” items marked “C," 
and M E" itcma marked with something other than “E° is in¬ 
dicative of the extent to which die pupil rend* toga beyond the 
lata A relatively low score would be indicative of relatively 
{icater maturity in thinking with data. Such analysis ia one 
af the characteristics of the handling of tho test data in the 
Eight Year Study of thb Progressive Education Association 
and In the Cooperative Study in General Education of the 
American Council on Education, The following atatement ap¬ 
peals m a recent report of the Board of Examinations of The 
University of Chicago. 


The modem tendency ii to enntnict the ,terns to that the wrong 
response* are wrong in a speafiewajf—for example, a definition when 
W too blued, or a definition whitb is inadequate. When alternative 
answers ore so cboren, a regular pattern of Incorrect response* u 
established which upon analysis yields much more information about 
the students' mental habits chan did incorrect responses constructed 
by the older methods. By meant of dm pattern type of analysis, It 
is powibfe to deicnnme whether ■rodents' erro" fn readme and inter¬ 
preting date cornut in nying that certain thugs are in the dare 
which hi fact are not, or in laying that “stain thing* or* not in the 
data which m fact are. It may be found that ituonti are able to 
iwct statement* which an* wuh the data presented but have diffi¬ 
culty with statement* which disagree with the data presented. 

It la difficult to wnu exercises of the type* described Such 
exercises mutt be written and used, however, if pupils am -to 
seek worth-while objectives and if the degree of attainment of 
worth-while objectives is to be measured Where measurement 
is restricted, objective* are also restricted' 

'When a teacher has written such exercises it {* essential to 
Meat* cirafub?T»lgstlon by other teachers who have the run 
■Mmol objective*. The best evaluating u done when the 


# 


Teach kb tost* 


H 9 


evaluating teacher does not have access to the key, but answers 
tho items herself Companion of several such evaluations u 
valuable in the rejection of bad items and in the revision of 
othen. For example, in a biological science examination the 
students were asked to mark certain items true or falsa on the 
baou of the "principles of inheritance M The following item 
waa accepted a* falae by several biological science instructor*; 
“Boys tend to resemble the father, while girls tend to resemble 
the mother " However, one instructor pointed out that in s 
certain important respeet boys always resemble their fathers 
and girl* always resemble their mothers In the directions pro* 
ceding the items the following qualifying phrase was added to 
take care of the situationi "excluding primary and secondary 
sex characteristics " 

The preceding paragraphs have dealt exclusively with ob¬ 
jective ex*raise* In any balanced program of testing some es¬ 
say exercise* should be included. In writing such exercises 
“fact" questions should be avoided. Objective exercise* con 
tret knowledge of fact* more efficiently and representatively 
than essay question*. Essay exercises should represent novel 
problematic situations For example, the following essay ex¬ 
ercise appeared in a recent physical science comprehensive- 
examination • 


in 1472, t-bnstopber Columbus began tns voyage of discovery by 
HUIUII toudmuft to the Canary Islands which are near the Coast of 
Africa and in latitude 28* N. He then continued hi* voyage by niL 
J? rl“ p *** fh* Atlantic Ocean between ibe equator 
and 30® N On his return trip to Spain, early in 1493, he finrt Vifid 
northeast unnl he waa Somewhat more than 30® N and than oiled 
cut to Spain. On the basia of inform its on riven In your physical 
aclenca tret, explain why Columbus aided as described above. 

Several blank lines followed this exercise In the test booklet 
Nothing Is said about Columbus in the physical Science text, 
but information on the belt* of the winds is given which could 
be applied by the student in responding to the question Co- J 
lumbui took advantage of the northeast trade* in his voyage 
to the New World and of the prevailing westerlies on hu rctw» ► 
to Spun. i| 

Essays have been baaed on selection* of quoted m*req»l-$ 
and on cartoons. In the field of English composition jt jjP%j 
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effective to hive miyi bwed on notes praented tot reading 
at the time of the examination, or prior to the time ol examine- 

"““information is needed for the tenting of a correct reaponse 
to eeaay eiemset which are thought question! or novel pr°b- 
bms but the qiuJlty of the teiponse will alio depend upon the 
extent to which the .indent critically analyeee the situation 
and thoughtfully otganttes ha inhumation in an effort to 
meet it. The ecorere ihould be leneitive to more than thecor- 
rectna* of the information presented by the student, Their 
nungt should be based not only on the correctness of the facte, 
but also upon evidences of superior selection, evaluation, and 
organization of the facts presented. Comparison of student 
responses with a scale of response* to the same or to a aunuar 
question may be found t<» be an effective procedure. Another 
possibility is the use of directions for scoring m which such 
characteristic* as organization and originality are defined and 
illustrated, and suggestions are made with respect to the weights 
to be given to each such characteristic 


AiuUysv of Test Scores 

After t tat has been given and scored the test data should 
be subjected to analysis Anslysis is essential if the teacher » 
to know the extent to which objectives are being attained 
One type of analysis has been referred to in the paragraph fa)- 
lowing the exercise baaed on a graph It is also very desirable 
to determine the per cent of correct response to each exercise 
for the group taking the teat A low per cent or correct re¬ 
sponse may indicate the need of further instruction In some 
cases low per cents of correcc response may warrant the rejec¬ 
tion of such items for use in testing subsequent classes, or the 
omission of such subject matter as inherently too difficult. 
The analysis may be extended further than merely determining 
per cents of correct response, One can, with a little labor, de¬ 
termine how well each exercise correlates with whatever is 
measured by the test as a whole. 

One way to do this type of analysis is to separate the papers 
into two groups The “uppergroup” contains all paper* above 


TEACBba TESTS 

the median ecore of the test ae a whole while the "lower group" 
contain! all papetawith total aeorea below the median mtal feet 
more Telling one ten paper at a time and opposite the num¬ 
ber! of the exercise, on a tally eheet, the teacher tallrel lor each 
exercise correctly answered For example, if the fim paper 
has correct answen for exera.ee 1,2, 5, 7, and to on, the teacher 
maltee a tally mark after thcee numbere on the tally iorm 
Another form a ilmrlarly prepared for the “lower group Ihe 
per cent! ef correct reeponM for each of the group! ate then 
computed, (Semples of 100 papere m each group avoid the 
necessity for such a conversion,) Since there are equal num- 
' bora of papsri in the upper and lower groups correspon ding per 
cent, may be averaged to obtain the par cent! of correct re¬ 
sponse for the entire group taking the teat The pet cent. lor 
the upper and lower groups are used in reading the correlation 
coefficient from the abac shown on page 122 

For eximple, suppose that 65 per cent of the upper group 
answer exercise 17 correctly while only 25 per cent of the lower 
group do so The correlation between suqeese or failure on the 
Item and the total score on the tesc is + .60 Such «n item 
makes a significant contribution to whatever as mounted by 
the teat as e whole. This correlation can be aeen in the follow¬ 
ing table which need not be constructed for each item, but 
which u useful in explatnmg the above 

FAHUWe OW ITXM SUCCESS £>H TOM 


35 

6 5 

75 

•25 


Taka the per cents 80 and 55 aa another example. Here 
the intellect does not fall on one of the lines in the chart. One 
mint interpolate between the lines labeled r«+ .40 and r*= 
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+ ,50. The r m ihi* care w approximately + 43 (One inter¬ 
polates along the imaginary line perpendicular to these curves,) 
When an exercise drops below + 20 in correlating with the 
total test score the inference is that the exercise is nor making 
a significant contribution to the rest The first time a c»re- 
ABAC FOR. TOMvTBST CORRELATION 


racsNTAos off remit oiocr r amino tkl him 



fully coasnictcd ie*t is given it is noc unusual to find that one- 
fourth or one-thud of the exercises drop below this value. 
Certain exercises may yield negative correlations. For ex¬ 
ample, suppose chat only 25 per cent of the upper group an¬ 
swered exercise 48 correctly while 65 per cent of the lower group 
responded correctly. The correUrion is —,60 and this nega¬ 
tive ndsthuuhip is illustrated by the following table* 


mow on inn success on item 


ABOVE UEXlAM 


BSL0W X SO IAN 


Low or negative correlations most often indicate that an 
exercise is bad The key may be wrong, the exercise may be 
ambiguously stated, or the exercise may be much too easy or 
much too difficult: Analysis of responses to other altematjyej 
can determine a better key when the exert we* are of the type 
requiring judgments; for example, the type of exercise illus¬ 
trated with reference to the reading of the graph of state expen¬ 
ditures In some cases a low or negative correlation does not 
necessarily mean that the exercise la a poor one. It is possible 
that the exercise u measuring some trait which is worth while m 
itself, but which is not related to whatever is measured by die 
test as a whole In every case the teacher should study the 
exercise in relation to the total per cent of correct response and 
the correlation coefficient, and should formulate judgments with 
respect to the merit of the exercise and the degree of attainment 
revealed by the data. 

The type of analysis just described applies to objective a- 
metres No definite procedure can be suggested for eesty ex¬ 
ercises When the scores on e given exercise range over several 
points, ordinary Pea non product moment coefficients can be 
calculated between the score on the exercise and the wore on die 
test It would seem much more useful, however, to analyze 
essay leaponsa in terms of some classification with respect to 
various types of merit or of limitations. 

The discussion in the preceding paragraphs may seem to be 
carrying the ‘'improvement of testa” ■ bit coo far for the dare- 
room teacher. It nay seem that an Inordinate amount of work 


15 

25 

35 

65 1 
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and cerebration is invoked If, however, teaching ip aa much 
worth doing aj the producing of manufactured articles or mate- 
nals where the processes are controlled from stage to stage, 
then the labor involved js more than justified The teacher 
who is willing to do these things will find that the “improve¬ 
ment of tests" is not merely that. It is also the improvement 
of teaching. 


PREDICTION OF COLLEGE SUCCESS BY MEANS OF 
THURSTONE'S PRIMARY ABILITIES TESTS* 


CHARLES a GOODMAN 
The Pmniyivula Sqm Collets 


In his 1940 monograph on factor analysis, Wolfla (18) 
states: "Few attempts have been made to put the results of 
factor analysis to practical use Most of these few have dealt 
with intelligence testing . Some studies, Chum (5) and 
Schneck (9), have indicated that achievement in individual 
courses can sometimes be predicted as well [as], or better, by 
special tests or by specialized subtena than by the total score 
of a general Intelligence test. Which testa will beat predict 
grades in individual courses is an empirical problem As a 
start toward answering that problem, Thuratone hu prepared 
a battery of sixteen tests, giving perception, number, verbal, 
space, memory, induction, and deduction scores (15). He 
warns (16) that this battery is still in the research stage and » 
not ready for routine use. It has been used fay Bemrcuier (3) 
and Stalnaker (11), but no data have been published to indi¬ 
cate how wall It answers the administrator's wish for better 
methods of predicting scholastic achievement.’* 

This paper it a report upon the studies conducted at The 
Pennsylvania State College specifically related to the possi¬ 
bilities of using the Thurctono Primary Abilities Tests for the 
purpose of predicting scholastic achievement, it provides, in 
some measure, a partial anewer to the statement made by 
Wolfle (IS) that there is a need for data to indicate how well 
these tests answer the administrator's wish for better methods 
of predicting scholastic achievement 

'Tbc writer wishes a scknmtcdsc ha flunks and tpprxcatfcn tn Minus* 


report «ba finding rf ihar atuiiis related tn tM* paper 
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Experiments end Findings 

Bemrcuier and Goodman (4) conducted a study in 1939 
to determine how well they could predict the success of fresh¬ 
men engineers at The Pennsylvania State College by means of 
the Thun tone Primary Mental Abilities Tests. One hundred 
seventy freshmen engineer! served as their subjects. Bern- 
reuter end Goodman then obtained correlations for the (our 
college courea of Chemistry, Drawings English Composition, 
Mathematics, and semester point average* with the Primary 
Abilities Tests. They then calculated multiple correlations, 
using various combinations of the abilities, with semester point 
average, Chemistry, English Composition, and Mathematics. 
The results are shown in Table 1. 


+ .49. In both cases the increase in the size of the correlation 
is slightly more than that obtained using the single abilities of 
reasoning with Mathematics, + 44, and verbal ability with 
English Composition, + .44 The best multiple correlation ob¬ 
tained, + .51, was with fint-semester point avenge, using the 
abilities of number, verbal, space, induction, and reasoning. A 
more detailed report of this work has been published in sn 
earlier paper by Bemreutor and Goodman (4) 

During 1940 Ball (l) administered tho Thurstone Primary 
Mental Abilities Tests to a group of 147 female freshmen and 
159 male freshmen attending the Liberal Arts School of 1)10 
Pennsylvania State College He then correlated their test 
scores for each of the seven primary abilities with the college 


TABU 1 

CottW**o»J tnJ tlmlAfU CbmbfffW ol it* Primmy AKElw «UA CaBift 
Court*! U SishoinMi 


TABLE 2 

CarrtUilont cf JkvmtMrft iHbw wdA Flrti SnM/Hr Petal Av*mt» rnnJ Crain 
n Ntae Limi Am CeOtg* Conriu 


T 

D 

NTS ID 

■MU. 

NgID 


+JX 

+JJ 

:s 

:s 

til 
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+ JM +.11 +21 -JK 

+2J +18 +21 +29 

+41 + II +21 +44 
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The highest single correlation obtained was + ,44 for reason¬ 
ing with Mathematics and + 44 for verbal ability with English 
Composition By using a combination of the number, space, 
induction, and reasoning abilities, and calculating a multiple 
correlation with Mathematics, they were able to obtain * mul¬ 
tiple correlation of +.49. In tbe case of English Composition, 
by combining the abilities of number, verbal, memory, Induc- 
tion, an d reasoning, they obtained a multiple correlation of 


Sonata post ■ voice,, 
Ait 





courea of Art, Botany, English Composition, French, History, 
Mathematics, Physical Science, Political Science, Zoology, aid 
with their first-semester point average. Ball's findings *re 
shown in Table 2 

It will be seen from Table 2 chat the range of correlation* 
found by Ball is from -.12 to +.41. The highest correlation 
obtained was + 41 for number ability with Mathematics Tbe 
next highest correlation, + .40, was that of verbal ability with ' 
English Composition. Verbal ability correlate* more highly % 
and positively with each of tho courses than does airy of ji 
other abflitia. The highest correlation Ball obtained for pgtfj 
of the abilma with Semester Point Average was+.35 with 
bal ability In b further effort to determine whether he tiM&i 
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increase his predictive possibilities with the Thuratone teste, 
Ball computed a multiple correlation of the memory, number, 
verbal, induction, and reasoning abilities, using as his criterion 
semester point average The multiple correlation was + 46. 
Dali's results show that optimum weights yielded only a slight 
increase in correlation over that obtained for cha single factor 
of verbal ability, which correlated + 35 with semester point 
average 

Etessemer (8), in a study similar to Ball's, attempted to 
determine the predictive possibilities of the Thuratone tests in 
the School of Chemistry and Physics at The Pennsylvania 
State College. Upon administering the AbdttUs Tests in 1942 
to 147 freshmen students, she correlated their test scores on the 
pnmaiy abilities with their first-semester point average and 
with the course of Inorganic Chemistry which each of her sub¬ 
jects had taken Inorganic Chemistry at The Pennsylvania 
Stare College is the first course in chemistry taken by freshmen 
students It is both intensive and extensive in scope, as can 
be seen from the following description in the College catalogue* 

Inorganic Chemistry (5).—The non metallic dements, fundamen¬ 
tal pnnb|>lea of the science arc studied In connection with the deicnp- 
tjTO themistry of nonrastalUc eJemenri and their compounds, prepares 
for future study of the idem*. Lecture 2 hours, recitation 2 hours, 
pricticum 3 hours 

The results obtained by Hessemer are presented in Table 3 
and show, as did Ball's results, verbal ahility to ba the best 
single predictor of semester point average. The best positive 
correlation, + .18, was that of reasoning with Inorganic Chem¬ 
istry This correlation is slightly larger than four times the 
size of its probable error. Interestingly enough, the largest 
correlation found was negative, - 25, for space with Inorganic 
Chemistry. When each of the primary abilities wm correlated 
with semester point average, the highest correlation obtained 
was that of + 44 with verbal ability, while reasoning correlated 
next highest, + 40 

In 1941 Trediclc (13) conducted a rather thorough study 
of the predictive poaabiliues of the Thuratone Prtmary Mental 
Abilities Tests and a battery of vocational guidance tests. Her 
guidance test battery consisted of the Otu, the Pressey, the 
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these abilities correlated with Semester Point Average was+ .51 
and again it was verbal ability. 

Tredtck then calculated the correlation# of the battery of 
vocational guidance tests taken by her subjects with the same 
five college courses which were used to obtain the correlation# 
with the Thuratone tests, and semesrer point average. The 
findings are shown in Table 5 and afford some interesting com¬ 
panions with those obtained with theThuratone A Wirier Tests 
TABLES 


frmUNwu at tU PnMbMit ChUmh tottery Gttov Oradu end Stma/ior 
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The highest correlation obtained for a single test of the 
vocational guidance battery with college grades was that of the 
Otis (Higher Examination, Form A), which correlated +.54 
with En glis h Composition. The next highest correlations ob¬ 
tained were for the Otxs with Home Economics 101, + 4B, and 
the Pressey with English Composition, +.48. It will be noted 
that both of the group tests for mental ability, the Otu and 
Prutey, correlate higher with these college courses than do any 
of the other testa of the vocational guidance battery. Further¬ 
more, the Otis correlates more highly with Semester Average, 
+ *53, than do any of the other rests of the vocational guidance 
battery. The Presiey correlates only slightly lower than the 
Otis with Semester Average, +.51. Comparing Table 4 with 
Table 5, it will be seen that the verbal factor la the only ability 
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Minnesota Paper Form Board, the Minnesota Clerical Test, 
the Meter-Seashore Art Judgment Ten, the Minnesota Assem¬ 
bly, and the Minnesota Spatial Relations Test . Her subjects 
consisted of 113 freshmen women m the Department of Home 
Economics at The Pennsylvania State College. After adminis¬ 
tering these teats she correlated the 1 seven abilities with the 
grades made by her subjects on the five college courses of Art, 
Chemistry, English Composition, Home Economies 101, Home 
Economics 109, and firat-semester Point Average. Her find¬ 
ings are shown in Table 4. 

It will be observed from Table 4 that the range of correla¬ 
tions is from - 02 for memory correlated with Art, to +.55 for 
verbal ability correlated with English Composition. It will 
be noted that Tredick's results, like Ball’s and Hcssemer's, 
showed verbal ability correlated more highly with each of the 
courses than did any of the other abilities The highest any of 
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that appears to correlate as well as the Otis or the Prcssey Tests 
with the various courses The Ofcr correlates slightly higher, 
+ .53, than does verbal ability, +SI, with semester point aver¬ 
age. However, the correlation with semester point average of 
the Pressey is the same, + Jl, as that for verbal ability 

In an effort to determine the relationship between her sub- 
jets’ scores on the Primary Abilities Tests and their scores on 
the vocational guidance battery, Tredick calculated the corre¬ 
lations for each of the tests Her results are shown m Table 6, 
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The data of Table 6 offer some indications why the correla¬ 
tions of Thurstone’t verbal ability With the five college courses 
are so similar in size to those correlations obtained for the 
Pressey and Otis with the same college counts. Of the seven 
abilities, verba! ability has the highest correlation of the entire 
table, + .76, with the Otis Test The next highest correlation, 
+ .68, also involves verbal ability with the Pressey It would 
appear from the correlations that both the Pressey and the Otis 
contain materials similar to those found in Tburatuno’s verbal 
ability It also seems thac the matenali of the reasoning, in¬ 
duction, and the perceptual factors overlap the materials of 
the Otis and Pressey Tests, w can be seen from the correlations. 
It may be that the perception ebility {a related to the "speed 
factor found to operate in tuned teats such as the Otis and 
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Praise y Memory ability correlates only slightly with the Otis 
and the Preisey, Induction and reasoning abilities correlate 
highest with the Revised Minnesota Paper F cm Board, The 
primary abilities appear to correlate only slightly with the 
Minnesota Assembly Test, The Minnesota Spatial Relations 
Test correlate* best of all with perception ability, but space and 
induction also appear to overlap to some degree with this test. 
On the Minnesota Clerical Test, Nome Checking correlates 
highest with the perception and number abilities. On Number 
Checking, the five abilities of verba], space, memory, induction, 
and deduction do not appear to be related On the other hand, 
fourof these five abilities, with the exception of momory, appear 
to be related. 

Finally, Tredick combined the abilities of number, verbal, 
induction, and deduction, using as her criterion semester point 
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average, and obtained an Jl of + ,61. Similarly, she combined 
tho Otis, prtssey, Minnesota Paper Farm Board, and Number 
Checking of the vocational guidance battery, using as her cri¬ 
terion semester pouu average, and obtained an R of + 57. 

Goodman (7) made a second attempt in 1940 to determine 
the predictive value of the Abilities Test In determining college 
success for his engineer subjects at the completion of their firat 
year m college Each of the factors was correlated with the 
first year's semester point average. The result! are shown in 
Table 7 

The abilities of reasoning, induction, verbal, space, and 
number were then used in a multiple correlation with first 
year's semester point average and the combined variables 
yielded a multiple correlation of + 49, which is slightly lower 
than that of + 51 obtained for the firat semester 

Goodman (7) then sought to determine whether be could 
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obtain better predictive values for his engineer subjects by 
using the sixteen Thurstons tests individually. These reiti, 
when combined, yield the score that is the measure Tor the abil¬ 
ity. According to Thurstore (14), each of the tests designed 
to measure the particular ability is highly saturated with that 
ability. The correlations between the testa that measure these 
abilities are shown m Table 8, The tests of the three abilities 
of number, verbal, and space appear to correlate highly with 
each other, while the correlations of the teats of the other abili¬ 
ties descend in size. 

TABLE H 
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The abilities testa were than correlated with the first year's 
semester point average and the zero-order coefficients were ob¬ 
tained. For the purpose of comparison, the zero-order coeffi¬ 
cients for tho tests and the ability zero-order coefficients are 
given in Table 9 The following facts are to be notedi both 
testa or the perceptual factor correlate only slightly with the 
criterion as does the factor itself, but it can be seen that the 
Identical Forms Test alone correlates higher than the composite 
of tho ability tests, whilo the correlation for the Verbal £num/r- 
ation Test is low in value. In number ability the r for the 
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single test of Addition » as high as the r for the factor itselF and 
the Multiplication Test r is not much smaller In the verbal 
ability, the Completion Test ft* again almost a# high as the abil¬ 
ity r. In space, the correlation for the ten of Cards alone ia 
greater than that for the ability, while the r for the second 
test of Figures is of low value. Both tests of memory cor¬ 
relate very slightly with the criterion while the r for the ability 
itself is slightly greater than the r for the Word Number Test, 
The three tests of induction have f*a of appimumattly die same 


TABLE 9 

ZuuOHm Yr aI Ike Sfsinu Tkununta Tuts witi tie CrU*vs* 


tun, 

Tat 

Fictor r 
•fab Oita wo 

Tar r wnh 

P 

Identical Fora* 

Verbal Esmmnnon 

+xe 

+ J» 

+ B3 

N 

V 

S 

Addition 

Mninpintwa 

Camphtma 

iMHWMia 

OnO 

Frgtmi 

+46 

+44 

+ II 

+ 46 
+44 
+ JI 
+ 40 
+43 
t 10 

M 

Word Nunbox 

Inimb 

+41 

+ 10 
+ B8 

I 

l-m«f Groopmf 

Mark* 

Number Falter* 

+41 

+48 
+ 40 
+44 

D 

Andunuk 

Number Sow 
Mrrbxnial Mavasma 

+4fi 

+J3 
+43 
+ 10 


wilt, while the r for the ability ik somewhat laiger than that for 
any of the teats In reasoning, the r for the test of Arithmetic 
is almost as large as the t for the ability, with the Number Senes 
Test r slightly smaller. The tart of Meehantcel Movement* 
correlates slightly with the criterion and with the other two 
rests of the ability, Upon obtaining the aero-order coefficients 
between the teats and the criterion, u pydtiple correlation of 
eleven o| the tests was computed The eleven tern were 
rejected which correlated highest with the criterion and lowest 
with, the other rest*. The tests selected were Arithmetic, 
Number SitUr, Seme-Opfashes, Completion, Letter Group-,. 


mg, Addition, Marks, Number Patterns, Cards, Multiplication, 
and Word Number The mu topic correlation obtained, using 
these eleven teats, was + 48 

It was apparent that there whs little hope of raising the 
multiple coefficient by adding the remaining five testa, since 
they correlated very little with the criterion It will l» noted 
that with the eleven teats the correlation was no greater than 
chat obtained by using only five of the abilities 

White (17) in 1942 made a study of the Thurstone tests 
and college prediction in Home Economics, using 94 1 of die 
same 113 subjects used by Tredick. At the time Tredick con¬ 
ducted her study these subjects were freshmen and had com¬ 
pleted their first semester's work. When White conducted her 
study, the 94 subjects she used of Tredick's group were then 
sophomores. White combined the grades for all of the Indi¬ 
vidual courses taken by her subject! and took aa the represen¬ 
tative score the students 1 avenge for each of the particular 
typos of courses. Semester point average in White's study 
represent* the average of two years of college work by her sub¬ 
jects Correlations were then obtained between each of tba 
primary abilities and tho average score for work id Art, Science, 
Home Economics, and semester pomt average. The results 
are shown in Table 10 

It is worth comparing the results obtained by White *1 
shown in Table 10 with those qf Tredick in Tabic 4. White's 
data showed that each of the abilities correlated lower in ill 
cates but one with semester pomt average than those obtained 
by Tredick. The exception is reasoning ability, which corre¬ 
lates + .45 with semester pomt avenge and indicates an increase 
over the correlation of + 42 obtained by Tredick, 

The correlations in White's study for the abilities with Art 
average am generally higher than those found by Tredick* 
For Science average the /s of the abilities calculated by White 
are mixed, some being higher and others lower than thoee 
Tredick obtained for the abilities with Chemistry White's 
correlations are all higher with English average than those cat- 
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ciliated by Tredick, with the exception of induction, which is 
(lightly higher for Tredick All of White’s f's with Home Eco¬ 
nomics average are lower, with the one exception of induction, 
when compared with Tredick's correlations for Home Economics 
101 and 109 Another fact worth noting is that in White’s 
study all of the correlations of reasoning ability are highor than 
those obtained by Tredick. It may be that reasoning ability 
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becomes more important when, presumably, the degree of diffi¬ 
culty of college work increases One mure fact worth noting 
u that White's correlation of verbal ability with English aver¬ 
age was + .65, while Tredick obtained an r of + J55 for verbal 
ability with English Composition 

White then correlated each of the sixteen Thurstone tests 
with the average Tor each of the four courses and semester pome 
average. These results are also shown in Table 10. Twenty- 


two per cent of the correlations for the single teats in Table 10 
are higher than, or equal to, the correlations for the ability 
In many cases the correlations of the tests of the ability are 
only slightly smaller than correlation for the ability Finally, 
White computed multiple correlations, using first, combina¬ 
tions of the abilities, and secondly, the individual teats When 
she combined number, verbal, memoiy, induction, and reason¬ 
ing abilities with semester point average, she obtained a mul¬ 
tiple correlation of + 59 Combining number, verbal, and 
reasoning abilities and correlating them with semester point 
average, she also obtained a multiple correlation of 
Lastly, she combined the five individual teats of Completion, 
Sam*-OpponiAi, Arithmetic, Number Senes, and Addition, and 
obtained a multiple correlation of -i- .62 

Other Thurston e Studies 

A number of other studies have been reported on the pre¬ 
dictive possibilities of the Thurstone Primary Mental Abilities 
Tests, Yum (19), in a.itudy at the University of Chicago, cal¬ 
culated multiple correlations using various combinations of the 
abilities with semester average The best multiple correlation 
he obtained was + ,422 using all of the abilities, Shauner and 
Kuder (10) have reported correlations of the abilities with 
average grades for tho 1938 freshman class of the University 
of Chicago Tlie highest correlation reported for average 
grades was with verbal ability, + .415. Correlations are also 
reported by these writers for the abditics with four introduc¬ 
tory courses at the University of Chicago Deduction corre¬ 
lated with Biological Science, + ,418; verbal ability, + 472 with 
Humanities, deduction, + 485 with Physical Sciences, deduc¬ 
tion, + 427 with Social Sciences Multiple correlations using 
all seven of the abilities with the four introductory courses 
yielded the following it's. + .500 with Biological Sciences, + 541 
with Humanities, + J61 with Physical Sciences, + 566 with 
Social Sciences. ( 

In 1941 Ellison and Edgerton (6) tested a group of 49 stu¬ 
dents at Ohio State Umveraity with the Thurstone Primary 
Mental Abilities Tests, The highest correlation obtained, 
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+ 44, was for verbal ability with point hour ratio The remain- 
lug ability correlations with point hour ratio range from - .24 
to * Jl, Combining ths seven abilities, they obtained an R 
of + ,640 with point hour ratio Ellison and Edgerton also 
report correlations for the abilities with grades m college sub¬ 
jects The highest correlations reported were, verbal abdity 
with English, + 75, verbal ability with Science, + 68; induction 
with Foreign Language, + 78, reasoning with Psychology, + ,63. 
They state "that the results can only be taken as suggestive 
and not as facts from which broad generalizations may be 
drawn” 

On the basis of the studies reported in this paper, the fol¬ 
lowing conclusions appear to be justified. 

1 The Thurstone Primary Abilities Tests correlate, on the 
whole, hh well u most standardized intelligence teats with cri¬ 
teria of college success. 

2. The Thurstone Primary Abilities correlate with indi¬ 
vidual college courses to same degree and can be used for pre¬ 
diction of success id these courses 

3. Verbal ability correlates higher than any other of the 
abilities with semester point average and individual college 
count* 

4 Multiple correlations obtained by using various combina¬ 
tions of the primary abilities yield some increase m correlation 
aver those obtained by using single abilities, when correlated 
with semester point average. 

5. Multiple correlations using various combinations of the 
■ingle tata that measure the primary abilities yield correlations 
with semester paint average that were in some coses higher than 
or equal to those obtained using the abilities 

6. Verbal ability correlates highly with tho Otis and Pressey 

tttti, and appeare to be overlapping some of the functions in 
these general intelligence tests. v 

7 The Otis and Press*? teste appear to contain somo of the 
same (unctions as those measured by the seven Thuratone pri¬ 
mary abilities, 

B. The single tests measuring an ability in some instances 
correlated highly with each other. 


9 A single test of an abdity will in some instances correlate 
higher with the criterion, than does tho composite of revenl 
tests of the ability itself. 
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TEST CONSTRUCTION IN PUBLIC PERSONNEL 
ADMINISTRATION 

DOROTHY C. ADKINS 


Introduction 

When the public is persuaded chat positions In the public 
service should be filled by the beat qualified persona and a- 
presses its conviction through a civil service law, a tremendous 
responsibility, that of predicting which persons actually are the 
best qualified, devolves upon the agency charged with adminis¬ 
tering the law Increased emphasis on the impartiality of the 
selection of public officials has been accompanied by growing 
reliance on examining processes that are objective. Hence, 
the major attention of this article is devoted to topics relating 
to the construction of the written examination in civil service 
In this interpretation, probloms common to the academic set¬ 
ting have been largely excluded. Although the article is further 
restricted to problems arising in state civil service or merit sys¬ 
tem jurisdictions, certain of the comments may apply equally 
to civil semee at the federal level 

However obscured it may be in practice, the essential of any 
merit examination is that it predict efficiency on the job Those 
who are not likely to perform satisfactorily on the job should 
be excluded from the final list of eligibles, and those who achieve 
places on the register should be ranked in the order or predicted 
job performance. To these ends each part of the total examina¬ 
tion process should contribute 

State civil service examinations often include, in addition to 
a written teat, a rating of training and experience and an orri 
interview If proficiency in the operation of machines or equip, 
merit is essential, a performance test may be one of the com- 
Hi 
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ponents, Far some positions phases of the examination may 
well be modified or omitted, but usually not the written 
examination 

/fating of Education and Experience 

Most stste jurisdictions determine initially who shall be 
admitted to examinations by prescribing minimum qualifica¬ 
tions of education and experience. Within limits, additional 
education may be substituted for a part of the experience, and 
vice vena. The process thus screens fine those candidates 
who do not meet the minimum qualifications. Those barely 
meeting the education and experience requirements may be as¬ 
signed the lowest passing score and those surpassing the mini¬ 
mum requirements higher ones, the scores depending upon such 
factors as amount, pertinency, and recency. The inclusion o{ 
ratings of training and experience as one pare of the examina¬ 
tion process is based on the assumption that differences among 
the candidates on these factors will be reflected in job per¬ 
formance. 

The argument (hat the entire burden of ths examination 
process should be placed on such ratings n untenable, however, 
aince the rating procedure is relatively subjective and unreliable 
and fails to tike cognizance of variations in the degree and ex¬ 
tent of knowledges and abdldes that const even among those oT 
similar training and experience. Such variations often an 
of greater significance in the prediction of job performance than 
are differences among candidates in education and experience 
beyond the entrance requirements Further, if these require¬ 
ments are relatively high, differentiation among candidates 
meeting them may, by means of disproportionate weighting of 
mere survival, gjve undue advantage to those who have faded 
to progress. 

Thus If, in relation to the salary level and labor market, 
minimum requirements for a professional dais are low, say col¬ 
lege graduation, then assignment of a high educartoa-experi- 
ence score to a candidate with two year? qf pertinent graduate 
*bd three yean of dotely relared experience probably 
W«ild Contribute W the validity of the total examination But 
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if for a similar dasa entrance requirements are higher, say two 
yeara of pertinent graduate education and three years of experi¬ 
ence, then giving a higher score to a candidate with five yean of 
experience than to a candidate barely meeting the mmimum 
qualifications may be of questionable value, The principal 
effect of such practice in this instance might be to give unwar-, 
ranted higher scores to candidates who, although older, have 
not advanced beyond younger candidates also meeting the high 
requirements for the dais Moreover, the rating of training 
and experience don not provide an evaluation of personality 
differences ' 

Th Oral Interview ( 

The oral interview, despite its recognized limitations, seems 
to be the brat available instrument for appraising personality 
characteristics For civil service examinations little or no re¬ 
liance can be placed on papor-and-penc3 teats of peiaonalhy, 
which would fail to elicit frank answers in a competitive situa¬ 
tion with jobs at stake Behavior in an oral interview can alio 
be faked. For this reason, as well as unreliable rating, the oral 
interview included In the final score is usually weighted consid¬ 
erably less than are the other two parti, even though it con¬ 
tributes positively to validity, and the number of candidates 
failed on the oral is very small, 

Warranting at least passing mention are certain miscon¬ 
ceptions of what constitute desirable purposes of an oral inter¬ 
view, such as the idea that its aim is to reappraise the appli¬ 
cants education and experience or that it should be designed to 
ascertain the scope of the candidate’s knowledge or the degree 
of his general intellectual abilities, Since such factors can be 
measured more adequately by the other parts of the examina¬ 
tion process, the inclusion of a relatively unreliable rating of , 
them in the total score probably serves to lower not only tip 
reliability of the total composite hut, more critically, its 
validity. 

The performance Tut - 

Where differences in skill in operating machines is a perdr,' 
ndit factor, the total examination usually Includes a 
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uice test. For positions requiring such skill, the oral Inter- 

* is of negligible importance if the duties typically entail 
irttle contact with the public or with fellow employees Be¬ 
cause of this and not because they serve similar purposes, per¬ 
formance tests and oral interviews are rarely used for the same 
class Differential rating? of training and experience are also 
commonly omitted for machine-operator classes Performance 
tesu are costly, difficult to administer, urne-consuming, and 
frequently n* very reliable. But their use may add appre¬ 
ciably to the predictive efficiency of the composite. In view 
of their limited reliability and their failure to take adequately 
mlo account the ability to profit from further training and 
apenence, they are more appropriately regarded as a supple¬ 
ment to rather than a aubstituta for a written test. 

The Written Test 

Teil Content. Even though the other parts of the total 
lamination process may be bo important for certain classes 
that they should not ba dispensed with, no other is so significant 
or should bear so much weight as a well constructed written 
examination, The purpose of the written teat is to determine 
nhibly the extent of individual difference* m pertinent areas 
of knowledge and abilities Defining areas of knowledge and 
the abilities to be,sampled in an examination re achieved by 
means of job analysis, the results of which are commonly sum¬ 
marized in a clas9 specification Such a compendium, depend¬ 
ing upon its thoroughness and clarity, gives a picture of the 
duties of the class of positions, qualifications in terms of 
education and experience, the supervision exercised and re¬ 
ceived, ind the knowledges and abilities bearing on the duties 
of the class, This information should be supplemented by 
knowledge of the relationship of the class to the total organiza¬ 
tion and of the applicable! salary range First-hand acquain¬ 
tance with the job, although unfortunately not always avail¬ 
able, is of inestimable value. Since the written examination 
must contribute to the prediction of efficiency on a particular 
type of job, its most effective construction is contingent upon a 
clear idea of what the jab is. Its subject matter should first of 
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interpret die term practical to mean what they think the public 
means by practical, A “practical** anamination, then, becomes 
one that looks practical to the lay person, and, particularly, to 
the candidate An examination may of course be satisfactorily 
valid from the point of view of its correlation with job perform¬ 
ance and yet fail to meet the criteria for "face validity” of 
thia type, In the interest of fostering public support of the 
civil service principle, construction of examinatiooe efficient in 
predicting job performance and also acceptable tn the public 
as practical is the desideratum- An examining body that limits 
itself to the kind of face validity under discussion is not in any 
seme adequately fulfilling its funccion; nevertheless, its general 
aim will in the long run be appreciably advanced if it achieves 
discriminating examinations that, at the same ume, have this 
type of validity 

Rigorous application of this criterion of practicality requires 
nnt only careful scrutiny of the broad areas of knowledge and 
abilities sampled in the test as a whole, and of the general area 
covered m each item, but also attention to the several in¬ 
dividual concepts contained in each item Sometimes attenuon 
must be focused on the individual word In a tent composed 
of multiple-choice items, for example, the candidate may not 
recall merely the question or just the question and the best 
aniwei^-obviously, he may not know which answer is intended 
to be beat But he may remember and criticize tome of che 
answen Intend*! to he "distrtctere,” "decoys," or "confusions/ 1 
ds they ire variously termed. 1 IF an item constructor inno¬ 
cently includes Socrates in a diBtracter, he should not be sur¬ 
prised if the examination is later publicized as absurdly 
unpractical because it inquires into the candidates' Greek 
philosophy Particularly Ip (ho cose of civil service agencies 
not far removed from the publics they serve, every examination 
item should be reviewed from the standpoint of how it might 
look m the public prints 

Speed versus Power Public opinion also has important 
bearing on other aspects of the examining process. For ex¬ 
ample, a ]■ more favorable to a power test than to a test tha 
place* a premium On speed,' People say job duties are not per 


all he related to che prediction of job performance; and the 
weighu to be asngned different areas of subject matter should 
depend upon their contribution to the prediction of efficiency on 
the job 

The types of subject matter included in a teat have all too 
often been limited by lack of facilities essential lot item con¬ 
struction Administrators and test technicians place too little 
reliance on persons thoroughly familiar with the subject matter. 
Test technicians sometimes delude even themselves into believ¬ 
ing that they can construct or at least assemble an adequate 
examination in a specialized field without assistance. Much of 
the criticism directed at written tests w attributable to action 
based upon just such misguided self-confidence. On the other 
hand, examining agencies should not vest sole responnbllity 
for either item construction or assembling examinations m 
subject-matter consultants who are not skilled in examination 
techniques The sounder and more successful approach as¬ 
sumes collaboration between those schooled in content and 
those vened in technique, although an occasional agency may 
be fortunate enough to have on Its staff a penon thoroughly 
competent m both a subject-matter field end examining pro¬ 
cedures 

The content of the written examination should be related to 
realistic class specifications It is by necessity limited by the 
facilities for constructing or securing items A third factor 
that ennnot be ignored in its bearing on examination content 
u public opinion. Almost universally, civil service examina¬ 
tions are required to be "practical and related to the job,” al¬ 
though the exact statement of the criterion vanes It would 
■cem at first thought that estahliihment of a “satisfactory" 
relationship between teat scores and performance an the job 
would guarantee the meeting of this criterion—and the menu 
of such an argument will not ba denied. Pertinent to this con¬ 
tention, however, la the difficulty of establishing that the corre¬ 
lation of an examination with job performance is “satisfactory,” 
particularly in such a way that the public will accept the 
demonstration at proof of the “practicality" of the examination. 
Faced with tbit dilemma, the majority of civil service agencies 
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formed in a setting that emphasizes speed and competition. 
These in turn, they say, create anxiety and thus poor perform¬ 
ance. The public subscribes to the belief that '‘accuracy is 
more important than speed" without recognizing the corollaries 
that for many jobs, rapid is preferable to slow accuracy, and 
that speed and accuracy,' at least in relatively simple tasks, 
tend to be posmvely correlated 

Many candidates, and particularly those who have failed 
or attained low ratings on n time-limit test, shore the layman's 
preference for work-limit teats Some may believe thst speed 
tests unduly penalize che older, more experienced worker The 
fallacy in this argument is twofold first, although speed of per¬ 
formance of some functions, as measured by testa, probably 
does decrease somewhat with age, in reality the speed of the 
performance of the same candidate at different ages (within 
the usual age range of candidates but excluding the age or 
senility) does not usually differ markedly; and, second, if the 
speed of performance of particular functions did decrease ap¬ 
preciably with age, then, for those positions for which speed of 
performance of these functions is important, reflection of this 
decrease in the test score would increase the validity of predic¬ 
tion. Actually, companions referred to by candidates are usu¬ 
ally not baaed upon the performance of the same candidates at 
different age tavela but rather upon that of persons who differ 
not only in age but alio m basic abilities and who still would 
differ in abilities even if they were of the same ago The older 
candidates ihfty get lower eeoree on a speed test and thus appear 
to be penalized The score is not a result of aga hut or indica¬ 
tion of leaser ability., Reflect that most older candidates for a 
job at a level for which a speed test Is frequently given have 
worked for a number of yean without progressing as Jar as jobs 
which will be beginning jobs for che younger candidates Thus, 
among candidates for clerical positions, those Bged 55 do not 
represent the same kind of sample of 55-year-olds as those aged 
19 represent of 19-year-olds, 

Regardless or the false premises underlying this predilection 
for power testa, probably the best forecasts of performance in 
the higher-level positions will result from tests with time limits 
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m liberal that only habitual laggards will have difficulty in at¬ 
tempting all the items within the tune allotted, For the major¬ 
ity of clerical positions, however, where speed of performance 
clearly is an important component of job proficiency, achieve¬ 
ment of the best possible prediction now will tn the long run 
outweigh any advantage that might be gamed by a concession 
to public opinion at the moment The inclusion of a speed teat 
as one component of an examination, even for a clerical class, 
may wisely be accompanied by a campaign to educate the In¬ 
terested public on the reasons for its use. 

Length. Every person who has ever attended school baa 
opinions about examinations. Thus public opinion impinges 
on die proper length of a test. The school examination, taking 
from 30 to 50 minutes, is regarded as typifying the most trying 
ordeal to which a candidate should be subjected It is forgot¬ 
ten that normally the results of any one such school examina¬ 
tion sro considered along with the results of several others, < 
daily observation, and the appraisal of performance on many 
additional assignments, and, more important, that such ah ex¬ 
amination U designed to measure achievement in a relatively 
limited area of a single subject-matter field. In contrast, the 
civil service written test la supplemented, if at all, only by a 
rating of education and experience and possibly by a brief oral 
interview or a performance test; and it is designed to sample a 
much greater complexity of knowledges and abilities 

When candidates dearly realize this distinction, when they 
Understand why a test that adequately samples the major fields 
ml. ig to a job is more reliable and more valid than one re¬ 
stricted u coverage, they prefer tests sufficiently long to yield 
a reasonably just appraisal of their job potentialities Al¬ 
though candidates may complain (hat long examinations are 
endurance contests, experimental work on prolonged mental 
effort indicates that the affect is more potent than the effect. 
Unfortunately, some civil service jurisdictions capitulate to 
public pressures for short examinations instead of enlightening 
the public on the virtues of comprehensive sampling 

Type of Item Decision on the type of item best adapted 
to civil service tests should be weighed against adequacy of 
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all time Iurut, simplifies test administration and hence may 
contribute positively to teat validity. Thu is one way of over¬ 
coming the handicap of monitors poorly trained or too unstable 
to meet emergencies with poise snd skill. 

Repealed lhe of Items In the educational world test 
question! are sometimes used more than once, to the gratifica¬ 
tion of certain fraternities that keep files of questions available 
Scores tend to be higher on the second administration of a test 
Perhaps in college repeated use is not very serums, since a courts 
marie usually does not depend on one examination alone and 
since a mark in a single course may be of no great moment. 
Institutions that are placing dependence on comprehensive 
examinations and little if any an course marks, however, are 
increasingly'following the practice of using teat questions only 
once. Hie problem or maintaining the confidential nature of 
examination materials is even more urgent for civil service 
jurisdictions There may be organized efforts on the part of 
"cram schools” to obtain access to examination items used, 
some candidates may apply for the sole purpose of memorizing 
assigned portions of an examination In die absence of such 
studied attempts to sabotage a ment system, candidates never¬ 
theless may remember a few items uv detail and another group 
of question* to “look up.” Whether or not this factor would 
give any odds to these candidates or their friends if rite same 
items were repeated, some members of the public might ihsnk 
there would be an advantage. Thus the confidential nature 
of the examination fuses with the question of public relations. 
A civil service agency better maintains the support of the pub¬ 
lic it it can give assurance that no appreciable advantage could 
accrue to any candidate became of previous me of items. 

Statistical Analyses of Items. From the point of view of 
the refisbpity and validity of a test, inclusion of a few poor 
icons is negligible if the test as a whole is long enough. In 
one way, however, the problem created by indefensible items la 
greater in civil service tests than m almost any other kind 
This is trufl because a difference of one point may affect a can- 
dld*fo?* rank order and aim may determine his passing or foil¬ 
ing. Hence he may or may not get ajob. Since a difference 
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coverage of the pertinent areas of knowledge and abilities, 
objectivity of scoring, and ease of administration. All things 
considered, the use of a large number of objective items is in 
general preferable to reliance on an essay examination. In 
civil service, objective teats have largely superseded essay* 
agencies chat still use them, the tendency towards supplementa¬ 
tion by the objective type increases 

If broad, general essay questions are uped- m an effort to 
cover particular fields of subject matter, different candidates 
may not actually be answering the same question, a factor that 
may nullify attempts to place them in an order of ment If, on 
the other hand, essay questions are more pointed and limited 
in scope, the coverage or the requirements ib automatically 
restricted, although more reliable grading can be achieved 
The objective test not only has the advantage of permitting a 
broader sampling of pertinent knowledges and abilities than the 
essay test but also, if properly used, will almost certainly lead 
to markedly greater reliability in scoring. These are highly 
important advantages in civil service, where areas of subject 
matter to be covered are broad and where unreliability of scor¬ 
ing may have a greater effect upon human destinies than m 
almost any other field of testing. 

Test scoring should be objective and also as simple as pos¬ 
sible in the interests of efficiency. Wherever feasible, the same 
kind of item should be used throughout and the same weight 
assigned to eseh item. The multiple-choice form lends itself 
admirably to most purposes. Many forms that appear to differ 
are limply variants of this form. The argument that use of a 
variety of forms adds interest to a teat is insignificant, since the 
competitive setting supplies more than enough motivation 
Arousing interest in a teat per se is an empty gesture. Items, 
and even the several responses to an item, are sometimes dif¬ 
ferentially weighted in civil service testa, but there is growing 
awareness that unweighted and weighted scores for a large num¬ 
ber of items correlate so highly that little is to be gained by 
differential weighting. 

Use of items of a single, readily understood type, accom¬ 
panied by dear instructions to candidates and with one over- 
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of even one pant has this importance, great care should be 
exercised to exclude from a test items that might have a nega¬ 
tive validity The problem U of consequence, too, as it relates 
to public faith in the merit system A widely heralded appeal 
baaed on a few weak Items, which may really be of no great 
moment insofar as they affect the over-all rebabdity and valid¬ 
ity of measurement, can go a long way toward undermining 
public support of the ment principle. 

In speaking of the validity of a civil service test, we should 
have in mind the extent to which the test serves us basic pur¬ 
pose, prediction of performance on the job To estabbsh incon- 
trovertibly that a teat has validity for this purpose is of course 
difficult. II a candidate population is used, the fading candi¬ 
dates do not get Jobs, ao that there is no information regarding 
their work, and the distribution of job performance indices » 
thus curtailed to an indeterminate extent. Even more trouble¬ 
some ii the notorious unreliability of service ratings commonly 
used to appraise success on the job Made by a number of 
ratere, with individual standards, on employees performing for 
varying periods on jobs that differ although grouped under one 
classification, service ratings are sensitive to many uncontrolled 
and uncontrollable factors Too frequently clearly recogniz¬ 
able differences in job performance are obscured by a tendency 
to rate employees in the "above average” categories, with the 
result that distributions of ratings exhibit marked negative 
skewness And this is only one of the frailties to which the 
rater is heir! Unfortunately it is almost impossible to state 
just how unreliable service ratutgs are, smes conditions for a 
really crucial experiment in this area have not been met In prac¬ 
tice. Probably, however, .30 to .50 represents a safei estimate 
of the Pearson correlation coefficient between two completely 
independent sets of ratings, made by raters assumed equally 
familiar with the work of employees on jobs wnhm the same 
class, using rating forms developed with an ordinary amount of 
skill and care A similar correlation of ratings made specifi¬ 
cally as a criterion against which to judge a teat would tend 
to be higher. In any case, problems,of securing as a criterion 
reliable evaluations of job performance on a population of Wf^ 
ficient size are very great and In moat instances insurmountable 
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Etublishmejit of test validity /or a pamcuJ or group 0 f can¬ 
didate* ii troublesome enough, but efforts to prevalidate on one 
poup a te*t ro be used on another group for selective purposes 
arc more hazardous still 

If a population of employees rather than candidates is used 
the problem of proportionate representation of those who would 
foil the test If they were candidates remains Assurance is 
lacking that an employee group, for purposes of a validity ex¬ 
periment, is representative even of the passers among a candi¬ 
date group Parts of a test that are satisfactorily valid for 
a candidate group may not differentiate among employees, who 
have had experience on the job and who may all have learned 
to do certain kinds of tasks that only the abler of the candi¬ 
dates could perform Moreover, if the confidential nature of 
tests Is to be maintained, an agency may not wish to be in the 
pottoon of repeating, either wholly or in part, a test from which 
there n»y have been "leakage." 

Attempts to secure an experimental grdup of subjects who 
ire neither candidate* nor employees face these difficulties as 
welt as the impossibility of obtaining any measure of job per¬ 
formance. * 

Confronted with these obstacles, the experimenter in the 
field of civil service teats usually resorts to a candidate popula¬ 
tion and an internal criterion, which is customarily the total 
■core on a test composed of items sampling a variety of areas 
of knowledge and abilities Conclusions should be drawn only 
charily from analyses based on BUch a multifactored criterion. 
Since different areas are represented in the va nance of the total 
scores to differing and usually unknown extents, dependent 
upon the variances and interrelations of the part scores, gen¬ 
eralisations on relative validities are apt to be inappropriate 
and misleading. The item-test coefficient must bo interpreted 
m relation to the item difficulty for the population in question. 
An item that fails to differentiate among candidates for one type 
of position may discriminate positively for another Any items 
with negative coefficients should be carefully scrutinized with a 
view to improving the item if its re-use Is contemplated and, in 
any event, to gaining insight into the charactjnatlca that may ‘ 
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iMd to negative coefficient, An item that coml.tes nega- 
■mjjFwith u “““fi 1 nntorion may, however, be positively 
related to the ultimate criterion to be predicted. 

For nuilnplMholce iteme, it la advantage to examine 
ImV r ™<5tienr, which may be regarded a, a 

kred of summary .tatnne, but alao tho relationah.p of each 
ption to the criterion Thie haa beep approached in various 
weye by correlating each choice with the enter™, by fari- . 
the mean cmenon who ltlM[ Mch c|)oi b * 

finding the proportion of pereons in each of two contianed 
entenon group! for each choice, or by come other method, de- 
pending upon the type of item index preferred With access so 
rtuehnd of information nr each choice, the examiner can often 

prove iteme and learn how to conetnict better onet Within 
certain limit!, which of the many variant forma of item-imaly.ii 
teehmquu one am .erne, relatively unimportant. Ideally, ,t 
may be preferable to compute for each choice for each item a 
corre ct™ coefficient (ether a bhenal, a pornt blaerial, or 
both) with a multivariate criterion' If intern,! centum m 
only a imnll number of iconic, tbl. cppro.ch i. entirely piac 
£j B '| hW “' '™‘ l " ° n " v " nl1 finnrireda or oven 

rimueande of item, are .variable, a deg™ of eta,hr,cal refine- 
mene may profitably ho eacrificed ,n the intent. of urnn* a 
u^ „ Pr ° P T'° n * h “ d - Particularly if frrdlme. for 

“' h ™ ,reh “ re I- thi. caee, a reaL.bly aalinfm- 

tnry technique may be to uae the tetrachonc eorrelarion coeffi- 
" . c " tenon fiwhotomized at the median. Here, 

S’. 1 ' w!a ,5* 0( t "!“ ™ Bnd m addition the proport,on. of 
candliUte. above and below che entenon median who .elect 
each choice For mat pmpooe., a practical mdex of item d.f- 
fitulqi u simply the percentage of candidate, who select the 
•>*« answer 

Some agencies that do not at the present time want to re- 
P*at ite ms nevertheless apply item analysis techniques with a 
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view to gaining insight into the characteristics of good and 
poor items, Both subject-matter consultants and test techni¬ 
cian! will profit from reviewing results of statistical analyses, 
even though the values may be in large part indirect 

As dsta accumulate over a period of years, it should be 
feasible, at least for classes attracting large numbers of candi¬ 
dates, to construct an examination largely from pretested items 
only a small number of which have been used before tn any 
one examination. The results of statistical item analysis 
thould not be applied blindly, for both the reliability and valid¬ 
ity of items may be altered strikingly by changes in the social 
milieu, A test item yslid at one time may be based on a con¬ 
cept that later becomes such common knowledge that even the 
poorest candidate answers it correctly Or it may binge on a 
concept that becomes outmoded For such reasons as these, 
even were atatiaticsl indices available for a group of item* from 
so large a number of different testa that the confidential nature 
of examinations would not be jeopardized by judicious repeti¬ 
tion, competent consultants 1 review or items shortly before their 
inclusion m a test would still be essential to sound teat con¬ 
struction. 

Perhaps m the not too distant future factor analyses of civil 
service tests Can dispel the problem created by the unknown 
contributions of several factors to tho variance of internal cri¬ 
terion scores Then items can be correlated with separate fac¬ 
tors instead of with a hodgepodge. Even so, the question of the 
appropriate weight for each factor in the composite could not 
be answered rigorously. The solution to this problem awaits 
development of a satisfactoiy external entenon 

Etiabhihmtnt of Critical Scares As those experienced in 
test construction know, even when statistical data are available 
the difficulty of an item cannot usually be predicted with high 
accuracy unless one has worked a great deal with the particular 
type of item and is predicting the difficulty for a group of can¬ 
didates having a known level of ability on the type of item In 
question Although tho prediction for each item may not be 
accurate, many civil service agendes assemble groups of items 
with s view to setting the passing point on an a pnari hails, 
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In aome casea, the law or nilei under which the agency operates 
make such a judgment mandatory In many others, where a 
fixed passing point is not so specified, the agency nevertheless 
considers the predetermination of passing points desirable, pri¬ 
marily from the point of view of public acceptance of the 70% 
passing point via the educational system Truce with this 
concept, hoary with tradition chough it is, is not entirely with¬ 
out compensation. 

Although the judgment of the appropriate difficulty or a 
test is admittedly hard to make, adherence to a fixed percentage 
of the total number of items as the passing point, where its uae 
is not clearly inappropriate to needs of operating agencies or 
unjust to candidates, puts the examining agency in a position 
to combat pressures for setting passing points so that particular 
candidates are passed. Thus the agency u better able to be 
impartial Not infrequently, however, even in agencies that 
have a fixed percentage passing point set by law, scores, are 
transmuted if an examination appears to have been so difficult 
that a register resulting from it would contain insufficient 
names to fill vacancies Usually a linear transformation is ap¬ 
plied so that some percentage score less than 70 becomes a 
derived score of 70 and, obviously, so that some ocher condition 
is satisfied, such as the original top score being equated to a 
score of, say, 9$ on the derived scale The most appropriate 
second condition vanes with the character of the original dis¬ 
tribution and with the desired properties of the distnbution 
of derived scores 

Although it may be considered advantageous to attempt to 
estimate test difficulty so accurately that no transformation 
of scores will be necessary, transmuting upward is considered 
preferable to transmuting downward, because candidates, while 
welcoming acorn seemingly higher thin actually attained, are 
reluctant to accept scores lower than the jiercentages of items 
answered correctly Moreover, a test that is difficult will dis¬ 
criminate among candidates better than one that la too easy 

Several factors m addition to the inherent complexities of 
estimating test difficulty magnify the problem for the civil 
service examiner. One » that the nature of the candidate 
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population vanes with the labor market) and the extent or 
change, as it pertains to the abilities and knowledges being 
measured, is not easy to forecast. It vanes with recruitment, 
again unpredietably, It vanes with changes in minimum 
qualifications and « a l«iy level, in 3 wa y M to ei P ec,, * ly 
bothersome if minimum qualifications are lowered at the Bame 
time that the ,«Ury level is increased, a combination not un¬ 
usual In a tight labor market. In fact, with the multiplicity of 
(acton operative, it ia surprising that results us satisfactory 
u those commonly achieved are possible. 

Sometime* effort is made to set the critical score at a 
"hreak" In the distribution, on the grounds that failing candi¬ 
dates will accept their bt more readily. Some evidence to 
bolster this point of visw probably could be adduced At 
tubes, however, the procedure seems to be based on the aiiump- 
rion that peculiar significance attaches to a segment of the 
rango within which no scores fall, This u nonsense. It i* 
doubtless possible to construct a test having scores with a 
bonodal distribution Whatever gaps appear in distributions 
of civil service test scores, howfcver, are due to chance, not to 
rational design If the number of candidates is sizeable, there 
will be no appreciable breaks in the proximity of any reasonable 
critical score, and no matter pdiere this score is set there Will be 
scores just below it. All things considered, an agency should 
face this fact tough-mindedly and not waste time in seeking 
breaks m score distributions or in creating them artificially 
Compiling Related $xominationj. A task almost peculiar 
to civil service examination construction it that of assembluig 
for a single administration examinations for sometimes as many 
as 35 or 40 classes, bur perhaps more typically Tor from five to 
15 If there are no common requirements of knowledges or 
abilities among classes for which examinations are to be held, 
then the examinations are simply assembled independently. 
If there are some requirements common to two or more classes 
the examinations ordinarily include some common items sam¬ 
pling the overlapping ones, particularly identical degrees of a 
given kind of knowledge or ability 

One argument for using overlapping items for related classes 
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ably the mutimum number of differendating items for two 
examinations for closely related classes, each cons is ting of 200 
items, should be In the neighborhood of 50 to 70 items. The 
optimum- number differs considerably from one pair of classes 
to another. 

Order of lumt for Related Eaaminatlonr, A special prob¬ 
lem of ordering Items arises when, say, IS examinations are 
encompassed in a booklet of, say, 1000 items, many of which are 
common to two or more classes, If they ire ordered strictly 
according to subject matter, the items in any one examination 
are scattered, hence candidates who have to skip several groups 
complain that the mechanics are confusing and may waste their 
tunc. On tho other hand, If the items are ordered in such a 
way as to minimise the number of "breaks'’ for the examina¬ 
tions having the largest nqmben of candidates, the items can 
no longer be ordered entirely by subject matter, nor can an 
ordering from easier to more difficult be strictly maintained. 

If administrative controls are adequate, perhaps the best 
solution is to Assemble for each candidate a booklet containing 
only the items perauawg V» the examination or group of ex¬ 
aminations he u talqng, Such a plan creates limitless con¬ 
fusion if the program ia improperly administered. 

A solution that'places a leaser burden on sound administra¬ 
tive control* it to retain the plan of assembling related ex¬ 
aminations in one booklet and, to compromise among thp 
objectives of arranging Items (1) in the order that minimizes 
breaks m each examination, (2) in the order that seems logical 
so far m subject matter is concerned, and (3) in order of dif- 
i ficulty. Perhaps the simplest way to arrive at an ordering 
reasonably satisfactory from there three points of view ja first 
to establish an order thaomminures breaks for the most "pop¬ 
ulous" daises, then to adjust the ordering by subject matter, 
then further to rearrange the items jo that for any single dau 
the more difficult items in any distinguishable area follow the 
easier liens. 

Selection and Traring of Subject-Matter Consultants, 
emphatii wu given to subject-matter consultation m 
cml »ervke test conttrocrion. Brief mention will now be made 


is that some candidates may want to take more than one 
examination, if the extent of overlapping is slight, such candi¬ 
dates may not complete some of the examinations m the time 
allowed To permit candidates to take several examinations 
for related classes in a single day (usually the maximum tune 
Tor administration of an examination program), the only alter¬ 
native to overlapping is shortening the total length of each 
examination, which is of course disadvantageous from the point 
of new of reliability Same jurisdictions etc in the other di¬ 
rection by constructing tests with so large n proportion of over¬ 
lap that differentiation among the tests » unreliable The 
effect of this error is that candidates who fad ona examination 
are likely to pare another for a higher class in die same senes, 
because of rhe chance element m the small number of differ¬ 
entiating items. 

Such a result Is not the reduetto ad abjurdvm of the ex¬ 
amination process that it may seem at lint glance. As a mat¬ 
ter of fact, identical examinations for classes that differ in 
degree rather than in kind would m some instances bo ap¬ 
propriate were it not for interpreting to the public several 
passing point standards On the other hafid, the same passing 
paints might be set far the written component of the examina¬ 
tions for several classes in a hierarchy, if the total examinations 
for the classes were differentiated on some other basis Such 
a practice has sometimes been followed by the United States 
Crvd Service Commission. For jurisdictions more limited in 
size of the public served, different written examinations for 
separate but related classes seem preferable, because some 
the candidates for each are likely to be placed in the same 
agency on the basis of the examinations. 

Roughly stated, the most useful principle for differentiating 
examinations for classes In a series la that the examinations 
should contain enough different items to reflect reliably the dis¬ 
similarities in requirements and enough common items to en¬ 
able candidates who so desire to take three or four examinations 
at one sitting A further important advantage in the use of 
common items »the economy effected in item construction and 
review. Although there Is no mle applicable to all cases, prob- 
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or some of the problems of using consultants in this capacity. 
Both selecting and training consultants are simplified for those 
examining agencies needing full-time consultants and having 
money co psy for them Mora typically agencies seek inten¬ 
sive but intermittent service*. 

Unfortunately chose who ire recognized as authorities in 
their fields are s!ao those who are likely to be employed in posi¬ 
tions from which release for temporary assignment elsewhere 
is not easy to obtain. On the other hand, even if an examining 
agency has a continuing job of sueh magnitude os to warrant a 
permanent assignment of a consultant, many of those who 
would be acceptable ae consultants are loathe to abandon front¬ 
line operations to undertake the cask of predicting the per¬ 
formance of others They hesitate to leave a position that thsy 
know and like Co pioneer in a job that they may find difficult 
to understand and that may offer no clear line of advancement 
Having taken this hurdle, they become dissatisfied if oppor¬ 
tunities (or contacts with others m their field are too litnued 
or if for some other reason they "go stale” on the job. Because 
of these (acton, some agencies tend to prefer making a series 
of temporary appointments, possibly on o part-time basis. 

Probably no single solution to the problem of selecting and 
training subject-matter consultants would meet all needs. 
Where it is appropriate and feasible, however, a plan of having 
a senior consultant <on a full-time and permanent basis and ad¬ 
ditional consultants on a part-time or temporary basis has the 
advantage of providing continuity to the examination program 
while at the same time securing a reflection of different points 
of view in the examination content. This plan. Also simplifies 
the training problem, because a permanent subject-matter con- ( 
suit an c who understands examination construction can inter¬ 
pret aspects of tins field to other consultants in theepme area., 
often more efficiently than can the psychometriclan 

Some persons who are sought as consultants and who may 
be recognised at authorities simply cannot he taught to con¬ 
struct examinations. Sometimes the difficulty Is pppsiwidy 
attributable to temperament, sometimes to pattern of abilities, 
mots often to t|u combined influence of both factors. To de* ( 
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■ jjjjt fl consul cant cannot b a taught to construct items may 
rtouwe only hrif on hour end again, for one who approaches 
Stask with' interest and enthusiasm, may take a month or 
Zrt Such Ineptitude w not peculiar to any held It can he 
m psychology, art, grammar, accounting, social work, 
hw music, and statistics The Fruitful solution to this problem 
lf to «t another consultant as tactfully as possible. 

Once a consultant is found who xs both interested in and 
gdept ar jtcm construction and test compilation, he should be 

ven interpretation on the desirable length of an examination, 
wring procedures, time limits, reliability and valid,ty, trans¬ 
mutation of scores, weighting of several components, and simi¬ 
lar concepts. Two of tho most common misconceptions of 
abject-matter consultants are confusion between the difficulty 
and the validity of an item and failure to differentiate between 
a teat as a predictive instrument and as a teaching device. 
Much attention must be given to overcoming those and similar 
fallacies related to examinations , 

Magnitude o/ tAa Task The final problem of which brief 
mention will be made is tho scope of the examining job to be 
fauwithm budgetary limitation b. Although monies allocated 
u the examining function in any area are rarely of staggering 
proportions, probably the public personnel agencies suffer the 
molt from paucity of funds Tho number and variety of jobs 
far which examinations arc to be constructed and the sizes of 
the populations to bo examined in normal times are m many m- 
ninces almost unbelievable. In a single jurisdiction hundreds 
of 1 it« may bo affected by a single examination program Yec 
die annual budget for the examining function probably would 
represent only a small fraction qf tbe sum allocatrd yearly to 
dM "supervision of reindeer in Alaska." Despite the handicaps 
of nadequste staff and limited facilities for research, the notable 
progress of the last decade places, on those in the field of 
measurement an obligation for continued effort toward the 
solution of the critical problems that remain 


RELATIONSHIP BETWEEN KUHLMANN-ANDERSON 
INTELLIGENCE TESTS IN GRADE 1. AND ACA¬ 
DEMIC ACHIEVEMENT IN GRADES 3 AND 4 • 

MILDRED M. ALLEN 

New Boelkfle Public School,, New Bodnfe, Now York 

Tuu study u pert of a doctoral research on the prediction 
of academic success of elementary school pupils by means of 
the Kuhlmann-Anderton Intelligence Tests. The putpose of 
this study was to determine the predictive value of the KuU~ 
mann-Anderion Intelligence Tests as o wholo when adminis¬ 
tered In Grade 1, in the fields of reading, arithmetic, and spell¬ 
ing in Grades 3 and 4 as measured by the New Stanford 
Achievement Test, 

Data 

The subjects for this study were three hundred and twenty- 
seven pupils from ten elementary schools in New Rochelle, 
New York, Complete test results of these pupils from the 
school yean 1936-37 to 1939-40 were used, Tire tests were 
the KuMmamo-Anderson Intelligence Test and the New Stan- 
Jord Achievement Test 

Procedure 

An alphabetical dais list of fourth-grade pupils (1939-40) 
by schools was obtained from the school census cleric, From 
■'this Hit pupils Were selected who were originally in the lint 
grade hi 1936-37. A checking and re-checkrag of all test re¬ 
sults dating from Grade 1,1936-37, and including the fourth 
grade of 1939—40 was made in order to select a group of pupils 
who had taken the complete battery of tests ai used in the 
present sfudy 

* Parr cf ■ itudr for i Doctor^ dlwmnon coopl*te4 at New York Unifovlty, 
GnSmts SAocl of Edunticn, 1940. 
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The Kuhlmenn-Andsrson Intelligence Test for Grade J 
(Second Semester) ,v/u administered in February, 1937, when 
the pupils wore midway through the first grade. Tho New 
Stanford Achievement Test (Primary Examination) F Z, 
was administered in April, 1939, near the close of the third 
grade, and the Advanced Examination, Form W, of the Bame 
test vu administered to the same pupils in the fourth grade 

m October, 1939. 

The Evhlmann-Anderson Intelligence Test was personally 
administered, scored, and re-ficored by the writer The Nero 
Stanford Achievement Test (Primary Examination) was ad¬ 
ministered In the elementary schools by tho principal or a 
teacher who had teat experience and teaming. Both principal 
aod teacher-examiner received Instructions for the administra¬ 
tion, scoring, and tabulation of test results from the writer 
The tests were scored, and double-checked by specified tea Ch¬ 
en ui the respective schools, by the principal, or by the teacher- 
examiner. The Nea Stanford Achievement Test (Advanced 
Examination ) was administered by the writer to all fourth 
grades in October, 1939 This test was scored and re-scored 
by an assistant under the direct supervision of the writer, 
Randjj and Interpretations 

The mental ability of the pupils used in this study is shown 
in TabU 1 Correlation coefficients between measures ob¬ 
tained from the Kvhlmonn-Anderson Intelligence Tm in 
Grade 1, and performance on the Nero Stanford Achievement 
Tests in Grades 3 and 4 are shown in Tables 2 and 3 


TABLE J 
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The mein I.Q’■ of the pupils indicate average ability for the 
group as a whole, with the range of I Q.’s showing the wide 


scatter of ability commonly found in heterogeneous grouping. 
The intelligence level of the pupils remains about the same in 
Grade 4. (These were the same pupils tested in Grade 1) 
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Table 2 reveals coefficients of correlation between the Kuhl- 
mann-Ander'jm measures and educational achievement rang¬ 
ing from -32> to .53, with twenty of the thirty coefficients bc- 
t tween 40 and -SO. In view of the fact that these coefficients 
have standard errors of from .03 to -OS, no significant differences 
among the coefficients are indicated. The 1Q, Pc,Av., (Per 
Cent of Average Development) and E-Q, all include one com¬ 
mon element, namely, the chronological age (CA), which la 
not included in die mental age (MA) score It may be noted 
that some penons regard as spurious correlations of ratios in¬ 
volving the tame variable denominators* The critical ratio 
for tho difference* of the coefficients of .53 (I Q. or Pc_Av,, and 
EA) and .67 (I Q. pr P&Av, and E.Q ) is 3.1. 

Whether one considers the MA, I.Q., or Pc,Av teems to 
make little difference since all yield substantially the same co¬ 
efficients with the various subteit rtiults on the Stanford Pri- 
vjTp GuStfcsi, totftoMiifc MnMi, p J74, 
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binary Achievement Test. One part of the Stanford Primary 
Achievement Test a predicted sa well from the varioua Kukf- 
monn-Anderson measures as any other part. The only variable 
derived from the Stanford Achievement Test which shows any 
significant differences m the coefficient* is the E.Q. An exami¬ 
nation of the lost row of Table 2 shows that the correlation 
between M.A and E Q is about the same as the other coeffi¬ 
cients in the table The relationships between both the EQ 
and I.Q., and E.Q, and Pc.Av, are substantially higher (40 
compared with 67 and 67) The critical ratio of the differ¬ 
ence between the correlation of M A and E.Q, and the cor¬ 
relation of I.Q. and E.Q. u 54 Thu indicates undeniable 
statistical significance of the difference, 

Excluding the coefficients involving E.Q., the median coef¬ 
ficient for Table 2 ja 44 While this indicates some degree of 
relationship between thtKuhUiusnn~And*rson scores and edu¬ 
cational achievement, it is by no meanB great enough to be of 
much value in predicting third-grade performance from the 
Kv/Umanfl-Anderron Intelligence Tests administered in Grade 
1. Due to differences in content material- of the earlier KvhL- 
mann-Aitdarion Test (non-verbal) in Grade I, and tho verbal 
content of the Stanford Primary Achievement Test in Grade 3, 
it appears that long-range predictions of acadonuc achievement 
would not be reliable To insure a greater degree of reliability, 
intelligence teats should be repeated annually, and preferably 
within the grade for which predictions are made, The coeffi¬ 
cient of alienation corresponding to an r of .44 is .8980, an in¬ 
dication that error* of prediction would bo reduced by only 
102 per cent by the use of tho Kuklmtsnn-Anderson Intelh- 
genee Test in Grade 1, instead of making predictions without 
the tests. The higher coefficient of Table 2, .53, between IQ. 
(or Pc Av) and EA- yields a h (coefficient of alienation) 1 of 
.8542, indicating a reduction in errors of prediction of only 14 68 
per cent better than chance 

The most predictable measure obtained from the Seta Stan¬ 
ford Achievement Test, Primary Examtitation, when predic¬ 
tions are made from the Kuhlmann-Andenon Intelligence 

■J P Sanford, piydumente MnJuJr. p 362 
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Test administered in Grade 1, seems to be the Educational 
Quotient (E.Q ) The r of 67 between I.Q ( 0 r Pc Av.) and 
E Q. yields a k (coefficient of alienation) of 7424 and indicates 
a reduction m error of prediction of 25 76 per cent better than 
chance The E.Q, however, does not give any indication of the 
actual level of achievement, but only of achievement compared 
With C A. It is significant, however, os an index of educational 
blightness In this study the PcJkv (Per cent of Average 
Development) doe* not appear to have any advantage (for 
predictive purposes) over the IQ or MA scores obtained 
from the same test 


Co*gici**it o} Condition Between KnMmiijn Anderwn IntdUwnca Tm 
««* in Grade J end Hew Stinfonl Juhknmuu 

EamJiutna, Form IV) Performeec* fa CraJa 4 


Stanford Achievement Ten—QratU 4 


Kuhlmann-AnA»nofl Tut—CW. J 


MA 


IQ. 


PcAt 


Pangnph 1 
WeM Henning 
Rejdini Avengs , 
Ambnuw BcmobIo* 
Anlhmmc Computitfon 
ArrthmstK Avenge , 

TSwI^isrets", .. 
Eduaifoml Age 
Educioanil Quotient 


J7 

JO 

JS 


I 


Table 3 u very similar to Table 2, except that in this case 
the relationships shown are those between the Kvhlmcnn- 
Anderson Intelligence Test for Grade 1, and achievement mea¬ 
sured at the beginning of Grade 4 The Advanced Exam iu> 
txon. Form W of the Nata Stanford Achievement Test was used 
in this instance. The intelligence test scores are the same but 
the achievement test scores were obtained approximately six 
months later (after a summer vacation) and were obtained 
, from a somewhat more advanced examination 

None of the coefficients in Table 3 are significantly different 
from the corresponding coefficients of Table 2. The general- 
izarions regarding Table 3 are the same as those drawn from 
Table 2 Excluding those coefficients revolving EQ,, the 
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range of the coefficients of correlation u from 30 to S6, with a 
median r of 48, which is higher hy only one a, of .(H than the 
median r of ,44 id Table 2 In this instance it is again evident 
that the EQ » the most predictable measure of educational 
achievement. The reduction m erroii of prediction dua to the 
magnitude of r is practically identical with the corresponding 
reductions in Grade 3. The M A, IQ., and PcAv are about 
equally effective for prediction There seems to be no superior¬ 
ity of the PcAv aver I.Q as a predictive measure 

A study of Table 3 shows a higher correlation between all 
indices of intelligence (MA, I.Q., and P&Av.) and arithmetic 
reasoning, than between the same indices of intelligence and any 
part of the reading tests, The correlation between the indices 
of Intelligence and arithmetic average is higher than the corre¬ 
lations between the came indices of intelligence and reading 
average. The most noticeable shift in prediction u that of the 
arithmetic scores 

Summary 

The most predictable measure obtained from the New Stan¬ 
ford Achievement Test, Primary Examination (Grade 3) and 
from the Advanced Examination (Grade 4) when predictions 
are made from the KiMmamyAnderson Intelligence Test for 
Grade 1, seems ta be the EQ. (r«= £7). Since E,Q, does not 
give any indication of the actual level of achievement, but only 
of achievement compared with CA, thn Is significant only at 
an index of educational brightness The PcAv In this instance 
docs iu>t seem to have any advantage (for predictive purpose*} 
over the I-Q. or MA scores obtained from tho Kuhlmarin- 
Anderaon Intelligence Test. All three indices ot mtdhgence 
(MA., IQ-, and PcAv ) are about equally effective for preduj- 
thm. 

Coefficients of correlation between the JCvhlmann-AndefJon 
measures in Grade 1 and educational achievement m Grade 3 
range from ,32 to .53, and between the same test (KvJdmann- 
Andersan fnttlhgtnce Test, Grade J) and educational achieve¬ 
ment in Qrado 4 from JQ to ,56- These low correlations indi¬ 
cate that long-range prediction* of educational achievement 
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baaed on only one group intelligence test in the first grade are 
highly questionable, 
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MEASUREMENT ABSTRACTS* 

Arthur, Greek "A Non Verbal Ten of Logical Thin Id a*.” Jcnmel el Consulting 
Pryckohv, VIU (1944), J3-34 

Hie author hit faimnUtcd a non-verbal ten of logical thinking. ilmHar in 
purpoia to the Kahs Block Design Test, hoc employing design* to be reproduced 
«kh plain and (tended card* in virion* colon The praWorn procured hr the 
pnlarvi em ef three kind*, [wm, trior, and lequnce Tentative, norm* miUuhcd 
oo the buii of molt* obtained from 300 subject! mdirau an looaau In average 
Kore Inun one age group to the next. CoiAertw Anne McNally 


Bcrgmtno, Gmuv and Spence, Kenneth W ' 
lurcment 11 PsychologicalReUt*. LI (1944), 


‘‘The Logic of Peyehnj 


A mathodobvcir inityni ot hum of the problem* of paytbopbyclcal mnnuo- 
" end of other aapect* of meeiurerecnt in peydulogv 1* pram ted from the 
’ '' ‘' '*■ ' uuion of the 


endpoint of onennfie emplrinim After a danmton of the methodological ft 
of reference, m which the neenuty for opiraitooil definition* from a payneal_.„ 
bin* ii Mruied. end a renew of certain prion pin of phyucal maiurwnept, an 
analytic of mythophrocal meaiuremroc u given- It b roochided that not only 
ihnuld the uk of vatkn* term* applicable to phyral iDcunnmest be damned 


In piychophyml measurement, but that aicarajement Id psychophywa 
ect op u ■' nchrique ki ite own rgbt,“ Lemma BouthBel 


Brown, Fred. "An Espmntmal Study of the Validity ud Reliability of din Brown 
Pengnalhy Inventory for Children," Journal el Psyaholagy, XVII (1944), 

The Brow* Personality Inventory for Children wu idmumtercd to 77 dmJafiy 
dagooMd neurotic boy* ud 100 tnnul buji between the l(a of 8 end IS and lit 
Gridea 4-9, tnelmlve, In order to demroin* whether I ho inventory would ddlirtn- 
Mte rrlpfaly and nmsntoirty between rariadjtnctd and normal children. Highly 
orufiani difference* between the two group* were found m cadi of the five cite- 
gorier of the Itumitnent. Thne dan were lupplemeDizd by two addinonal aperf- 
racnti with the penotulity loveotory, which rein] ted in high tut-retut comlstbnt 
and ranirtten dy high relielnlitie* for the aubDity of individual item*. Catherine 
due McNally 

Brawn, Fred "Cornparame Study of the Imclltxtoca of Jmrkh and Scandmaviin 
Kvadersartcn Children " Journal of Cenel IcFiyekolaty, LXIV (1944), 67-92. 
Three hundred and twenty three (1)1 male*, 192 (amala) isond mutation 
Scandinavian ud 324 (178 unite, 146 female*) icawd genenuon J»wi*h kluder- 
garren children In (he Minneapolis Public School* were tnted on the J916 Revuion 
el the Stanford-Binei Rlgnnnu control of age, eea, and wao-eCODonnc aura* m 
tdection of the date wu axamied Comparlsmi made between the performure 
of (he two group* oo general intelligence, burl age, ud vocabulary ahnw no ngnl 
Scant ddferesca However, thin appem to be ft differttvx when the nnoui *ub- 
nan ire countered Vatutron among occupational laveli a greater far each of the 

C itudied than the variation within bufrnduat occupanoiial level*, and the dif- 
t chat appear* between the two group* dicmere a* one pruts Irouj lower m 
upper nce iipamw l level* Jdfrum D Jfolmaa. 

* Edited by Fonret A, Krogrboiy 
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Cstull, Raymond B “An Objretire Teat of Charierar-Temperamenti II " Journal 

of SacM Psychology, XIX (1944), 99-113, 

TWe etndy encomparere three dot met resarchei the 8r*t dealing with a group 
d 6Q Khoal children, the ucond with 49 adult w o m en, and the third with 40 
Undent*. The objective of ihe nudy w*a M determine how far a personality trut 
mgmable to evnyday llo utuition* can be made to axyrat IneH alio id a mini 
aim bbonttur iirm non, objectively tronblt and iwnmug on great ecru plenty 
of aepantna The mula indicate that tbo Ckaracur-Ttaigeramant Test to formu¬ 
lated bat high coMbtancy, b ilmoal unaifrelated with nuelligcDce, and ihowi so 
rigatfeaet ten ddferenre m mean performance Cathenne Rant McNally 


frycftOoMOrta, VIU II9W), 2US-Z1U, 

Thh paper praaaita ■ tat he dctinnimog ilgnMimeo of difference* between 
■e*ne of rerepUe which are drawn from pouuray iltcwcd papulatkm*, tnor* iptdf- 
ioDy, ihcM tunas ■ Pniron Type lit dUmhodon feocrion The quantity 2np*,/ne 
(wfa<re t equal* th* mean equaled dmded by the variuree and ■ n the number of 
tut* In die ample), which dntnbute* taaaH » Qn Squire fur lay drgret* d 


t mean divided 
* degree* of 
woe* of di- 

... ......_____-ho range rf pouible Korea a from 

to Infinity When * lower thcnrericil eeore limit (c) eriiti wbkh I* not am, 
quantity (Mcan-c) abodd be wed uulcad d tbo mean in all ericdimn*. 
mnwy Pryi ‘ 


tk* nine at the tree moan, for two independent ample*, the larger man dr 
by the Muller mein, which doenbuta (aelf ai F for Inept and Inept dtgn 
freedom, may be referred ra th* f datnbutron nbla for taring iigmAcwara o 
fCroat* between mam. The cur auiuna that the range rrf pouihle Korea a 


. _ nl Tata and Their Uao.” ReoUa of Educational Xaitarch, 
(1944). 1-27 

thh roue mn *n i the Immure for (ho three yarn ending July, 1943 The 

K ida era included 

■ Warm □ “Brief Overview of the Penod “ _ 

Ethri L, ■Coirent Conitrucrien and Svahnilon el InuUlgeiice Tat*.” 
* Fnenuii, Fnnk S. "Application* of Inldligroce Tat*." 

4. Sell*, Saul B “Mormnitot and Prwikuro of Special Abflme*," 

3 Traakr, Arthur E. “Current Cowtrueiwn and Evdiulsw of Petaonality ud 


fi. DwUjy, Jr^m G. and Andinon, Gordon V “Apjdicitiom of Pcnmnlity and 

7 JtiwwxU, ^dnlTS^ Krognun, Mona, and Albert, Kathryn. * 

. jladjod* a the Study of Penonalfiy " 

1. Fowwr, Wiiten G, “Meuarcrocut of Fiychocdonitimial Growth* 


■— — “A Study ol tire Subtan In tbo Revised Stadord-Bmet L rod 

bt Jonmat el Ceneue FrycAefogy. LXIV (19M), 3-36. 

1H author (yvteaadally Invattigata the problnu of whether nr not the lub- 
t*m in the Revised Stanhrd-Binei tat* wan •pprapnaieij pfaced m reference to 
dBniliy She aobmuwi 210 Form L and 118 Form M SlonloriLBinels m three 
nuthri* of aaalyil** (1) cakolation el ncreutigo of iuccom* on each gnhiat, (2) 
eakiditm of the rririol ratio lor (he ddfetoico In poena a go d die i*mt mental 
*P group pnaroi each auhtcR, (3) a nanesHm ol th* peesnd merirod, where cam 
wm mix ted only ff they had taken croy auhiot withm a year level Uelng the 
ant methodI aha found that there are inroal I trait whara the tubteK* are of unequal 
aSreiTy However, In com paring her result* to thou of Bather, who eondnetsd ft 
jjronar etndy, ah* found that they do ms agree ga to which are of Unequal diSadtr 
Hu lut two mare refin*d method* of lulyin ihowed that hi general uhnm within 
*• e*nU lonl wo* id equal d««Jty. Miner* D, Potman. 
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Greenwood, Edward D, Snider, Henron L. and Senti, Mfitnn M “GonaUtlnii Be¬ 
tween (he Wedulcr Menial AbilkiT Scale, Form B, and Kent Emergency Tat 
(E-G-Y) Admmateivd to Army PcreorueL" American Journal of Ortho*//- 
tklatr, XIV (1944) J7I-173 

Two hundred n u Urifr uied army men won wren lha K'acfulvr He*tel dUriy 
Seale, Form B, and tin Kent Emergency Ten. E-O-T A* a rctult, a cod&cwOT 
of corrdaiKm of ,74i 03 wai band between th* Inal Standard Score LQ. of the 
|VeckJer Mental Ability Stela and ibe Kent EaergeMy Test LQ. Allowing for the 
abnormal gronp of men re whom the lem were given, tin* ronrirnoa wal con- 
■idered high The aaihua coodaded dut rite Kent Emergency Tail w» n inltable 
lntclll|*nc* teat m •liniuoua whsh do rare penult more airoarva rating. Catherine 
Anne AfcNaliy 

Gulbkian, Harold “A Cmino In (h* Theory of Menul Tata." Psyckemtlrrhe, 
VIII (1943). 223-24S 

An outlroe lor a comae In t«c theory fajireecntrd, together with a Uii of 
tealgorarou, problom, and ■ bibliography Id* enuna hu been given in the 
Piydrology Diparement of lha UniTerwty of Chieago. Ho malarial la procured 
in audhvc form at the pmeni tine bteauaa of til* bemud need for Ii-'- 


— la aim «f uti—, - 

tint tit organised body of material lor insnuctmal mm 
able to thou mi ere*ted (Cbnncay Piythomelriko.) 


a may ba readly avia 


Haqr, Wflliam A., Wit I eon. Ccd L and Hint*. Herbert L "Tiro Seim Tot in 
MQjtary Selection* Psychological fiiunqU (1944), 37-46. 

The rothoii cum pare the vanoot paper-and-pctul psychological tens ud tin 
peyridam mtervlew u med ra the nre-itwncinn feroanug proceu They Gad the 
ptychbtnc interview msihod non Bealbl*, more mcluijva and ewer to ad mm It ter 
Iratn the machonlca) vieyepomt, peycbologial tat precederi [i pare caunmcftl 
both in minpower ud tint* and ta batter inndardaad end more objccim. A* yat 
then b no final check ■■ to which h the prefoablq preegdnre, but it ■ their belief 
that a good pnchiint h g better mccnmg nntnnnait thin ■ good tea ud a 
beiur titan a poor paychntritr Miriam " *- 


mt b 
June 


nD fieiero, 


loaen, Qdtord E. "A Nomograph lor 
PntiomaliHa, VTU (1943), 2®-!lO 
Direction are giveu fop conitnicriiig a v 


-----—_ _ inrtplc nooumiapfa 

madliiM, which Ii entered with inforroitasi (mm the cumulative —--- - 

trihuDoo. It ^va a Uoear Inwpdlba witinn the flu utceml ctmtalnmg th* 
nwdiu (Cotmay Pnchometria.) 


Them ran. G B "The fironfiennee of Kit 
Psyciometsika, VHI0943), 211-222 
The ceefGncnti of nnk difference tor 


iriieanca of Buk Difference OoeSriaD of Comletimi" 


tbffottU levels of ilgnHbitien an oven lor F) ol 2 B 30. Mo*t Of tile vftl _ 

obtained fay tnntinnm id Ottf nbla of protubdiUa for vanmu vilna ol X<P 
CctnptcUcn of the** dan with three obtained by bur other methods Indicate* (hat 
qoa method yidd* value* more appropriate ihu three obtained Irani Old* 1 data for 
coe&ainti ilgmSaat at the .01 lerd for N't Iitnn 11 ta 2S. Htu method alio 
provide* a movement rouna of obaWng appraajoiJK valot* id adrigntg EgniS 
cut ittjig jOI lavri Tor ATr (bore 30. noao for cantfan In enldatua the rignlfi- 
cuce of codEcnanta ofatahMd from date involving tic nnktroi ii indlrawd, The 
article mntiadt* with icuJiruuerulicore u in tiudea (I melhore nf d rt o nnnuig m 
aig n t fao c e of nnk dWanaca cacSnoiia. (Courtesy PsytkometriktA 

Tinker, Mfiro A. 

Board Tat* 

The R * 


’Spaed, Poww, and Level in the Revued Mloacaon Papa Form 

__ Journal nf Ce-telk AycAofen UGV <1944), 93-97 

Revised Mimesot* PgM^Fonn Board Ttst wu adroitutterod tn 10) aol- 


a tn analyx* tit* cdiiloeuMp of the work Inrot method id n 
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nut mulling ui “ipend’' Kzctl to |t» uiuimiun nra* man™ [bjihuj a ibt» 
khb ,nd ihi rdaHon of boib d that to iiinoird rise mown mating m pwr 
uotta. 0p«d ind Iwd ««n win found to tiij mkptpdently A rniwr progor- 
dst of ihi power ran wu icawnted lor Iff rfaed «wi Icril, wjin »pwJ eoniwu* 
no* rdiBTBT mow mihe power iaw thin lerd. Th* inxlt Wie*m «uy » iltent 
Ntnblfaq berwren uuefljrree ind th* Rniiti Popor farm Board TuL CtiMat 
Am MtNitiy 

ffhenr. Bob*" J end G*)^ R “The Coocwt if 

ihdltT m Rtlitfea uP»twr PifMtn." PiytAwwl/Ma, VITT <IW3), W-JH 
It a ihmo thii ippmcha other ibia the lOttnu] eoralitonry method u 
•naitiwttrt rdlibihS ere enher fa* oiMwtnrj or Ini to the nmo rerun] 
multi. The commonly Jiunfam moapnui of i iingle futor throughout ibe tat 
Imu fa dagatf d. however Thi rrfuideratioo d »tot mida up of a tub-tan 
adi eoapMed of • different onhopowl Imw d!rcl««I thit.the mnniiiooa of * 
uaile tutor produetd in urnuoue rfdfoite of rdUbifty w«h i ncio of (a-JO/ 
(« -1) to the aimer miraiic. Spedil ddBenJna irakn Irom tfw» error In appUa- 
thn J ewTOt teduilnua m ihait t»n <x to tot bitann in dfacuaed ^ App^"- 
dm of due mm uuilri-ficbx nrac^i u Itm-mly iudfa cioea «trail 1/ dlfieultm 
m thit Held. Hu Iran tat roc&otUt ipprouhn VW u in upper Umh rather 
than lm^ind ippraichu Vtfn ■■ » low I*™ 1 ' 

tiding necoann lor in artr-anmitioii error fa the Kodtr-Rktirdeon Itwnrdi (W 
A uw method of ooliooi tub tare butd opno the fam-urt coi&tcnt » propped 
ui laitnmiy pudwd. Eitfc* fate nnr method or i ranplm uncr milwii » 
npnhil u the only proper ippnucfl to the probloa of tut idabdity, and the 
{ma-svlHJS! cotKomt u ilmflirty roannunded «i the proper ippmech for iicm 
irtilyw. (Cnnoy AyrWfrtte) 


tJu'fU Choki Group Jm shot IB*™ u « w 
*— of dim iroope, it the U, 8. Nani Traci nr 
,uZc-M fa mid teg “fifae poeltim* 


on a eaOiple populates, oaim 

Sown. Newport, R. L Hie tat WH uoaooaww a irawag iw« . 

•chfag op « par cant d the ptwp iff 417 ooranJ aobjjca, It wu UMitefirt^ 
fapiknpgout toe owthee, ruling ndy » per caau id ■ pwpd MS men w- 
duipd m unfit lor Ninl aerria for neairwmhlitiie rsuora Fnnherowre, ofi 

croup of 1*1 «hw« •aJtoftMd to tlw oi^tiou —- 

ttady Ut fieily idjatgidfirforaerrfcj'thi tut pleW up I9pu 
at the abnormal group- The aotlwn bond the lert’haiuIuM* ■ 
if indoptnept iw mhw *el^k /, Sleiitff 
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NEWS NOTES* 

An latttun on «to dent penaend «wk wie held on the, Lot Angela Cinm™ 
d du Ururemty af Cdfornu florin* the wwfc begin rang July 24, n uiun m 
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■raefc. U S.N *. m on leave Damir of Perunnel and 
a^B^dm. CoC, Dre# Ualreniir, wd ■ mborad at 


i, (1.1 Darcy B fituk,OiRA, Aaeocbto Pitfnm et ftiychn|o*y it thn 
tfnrwnny el fartrTlon l«vv). * Eamtret OOetf. Nml Unit, fmalamtui 
of Trahnafatj - 

„ ,£r. ^ 

teatteA Orauarrtra, ffADTCi, Naval Aw fltabnn , Jiefaoareile, FWi 

.ESM5SaS!-a35.15HS 

ea’WirhBKtao.D C. 


ANNOUNCEMENT 


Following an almost unanimous vote by the membership 
to the effect that the nomindcing ballot should be considered 
aa the final voting ballot, the following American College Per- 
tojmel AuocUtion officen were deiignatfid Cot the year 
1944-45- 

President: E. G. WiDlarmon,* Dean of Students, Univer¬ 
sity of Minnesota, Minneapolis, Minnesota 

Vice-Pres.* D D. Feder., Bureau of Naval Personnel, 
Washington, D. C 

Secretary! Thelma Mills,t Director of Student Affairs for 
Women, University of Missouri, Columbia, 
Missouri 

Treasureri W W. Blaeaaer,* Administrative Secretary of 
the Penonnei Council, University of Wiscunim, 
Madison 6, Wisconsin \ 

Members-atoLaigc of the Executive Council: 

J. L Betgstreaser, Dean of Students, College of the City of 
New York. 

A. J Brumbaugh, Dean of Students, University of Chicago, 
Chicago, Illinois. 

Helen G. Fisk, Associate Director, Western Penonnei Serv¬ 
ice, Pasadena, California 

Robert Hop pock, Professor of Education, New York Uni¬ 
versity, Washington Square, New York, New York. 

Either Lloyd-Jonea, Professor of Education, Teachers Col¬ 
lege, Columbia University, New York, New York. 


* Serving fa nnsf ym ot i wra-jw ttm, 
tThe Hcreniy b elected lot a mo-gar um (19 4 4 46) 



A TECHNIQUE for scale analysis 

WARD H GOODENQUOH, Scr, A.U4 
Ih tha pan two yean a considerable amount of rime has 
been devoted to scale analysis by the Research Branch, Infor¬ 
mation and Education Division (formerly Morale Service^ 
Division) of the War Department, where tho use of scales has 
been important in attitude and opinion research. 

The aim of this pflper is to make available to othen working 
with scales one of the techniques for scale analysis developed 
in the Research Branch The theory involved in this type of 
analyst* has been outlined elsewhere 1 and is not directly dealt 
with hens » 

Four specific techniques have been employed by the Re- 
miiEb Branch. AIL produce essentially the cunt results, di(- 
fflfiog only in the mechahics involved All have limitations 
Ollier of a rigorous or practical nature 
The first of these is the least squares method (Guttman, 
lam, “The Quantification of a Class of Attributes " The Pre- 
fatlo* of Personal Adjustment, Social Science Research Coun¬ 
cil Bulletin No 48, 1941). This method is too laborious to be 
mdily usable when one le dealing with more than a few items 
and categories 

A second method, the one generally used by the Research 
Branch, is to plot data on a Scalogram Board, invented for the 
purpose by Dr Louis Guctman This is an easily used system, 
especially if the sample is confined to 100 cases or less, which 
Is usually adequate; but it is less rigorous, since manipulations 
of the data are baaed largely on inspection. The board has the 
further difficulty that it is somewhat expensive to construct. 

i Getxmo. LouU. "A Bilk for Sollof Qua Hath* Dm * AmirUan SdekJaf 
fcWJtnw, K (19*4). 119-150. 

1*? 
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A third method, the tnalsconng and graphic technique, 
has been used in the work of the Department of Sociology and 
Anthropology at Cornell Univeraity as well as in the Research 
Branch. It has the advantage that it does not require a board, 
but is less flexible 

Finally, the tabulation technique, the one to be outlined 
here, was devised to enable scale analysis without a board, at 
the same time preserving its flexibility. It is also more rigor¬ 
ous than the ScaJogram Board method and can be used with 
any size sample. Its limitation is that, for practical uu, the 
relative rank order of categories for each item must be assumed 
in advance, which is not necessary with either the least squares 
or Scalogram Board technique The technique is easy to learn 
and as a process reveals clearly what is operationally involved 
in scale analysis. 

According to Gunman's definition,' the multivariate distri¬ 
bution of a set of qualitative items forma a scale for a popula¬ 
tion if the following conditions are satisfied 1 

(a) the items have sameness of content (that is, they form 
a universe of content), 

(h) each item it a simple function of scores derived from 
the distribution. 

Condition (a) is mainly deteimined by the nature of the 
problem an investigator is interested ul For example, suppose 
we are interested in students* expressions or adjustment to life 
at a particular college. For this purpose we might ask auch 
questions as. 

(1) How satisfied are you with college life! 

(2) Do you think that what you are learning is worth 
while or not? 

(3) What sort of a time are you having at college! 

We ask these questions because they are a sample from the 
universe of content which is the object of our investigation 
Having defined the umveree (in this case, adjustment to col¬ 
lege life) and having collected a sample of data, we want to 
know whether or not the responses we have obtained behave 

*7ui. 
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as simple functions of a single variable If so, this will be evi¬ 
dence chat the universe is a scale for the population, and we 
can denve meaningful relative adjustment Korea based on the 
responses to the items 

This brings us to condition (b) of a scale. Can we show 
that each item is Or is not a simple function of scores derived 
from the distribution of the items? The procedures to be dis¬ 
cussed here refer mainly to this problem, the practical one of 
outlining a specific technique whereby It is possible with rela¬ 
tive ease and precision to determlna whether or not a set of 
items satisfy the formal condition (b) of a scale. 

The question might anse aa to why this u a problem at all 
If a set of items formed a perfect scale, this would be eaBily 
established by any of the several techniques The problem 
arises when the scale-relationship of a set of items ir obscured 
by the item? categories This requires the combtntne of cate - 
gtmes, which manipulation creates the mBin technical problem, 
as wifi be Ken in the ensuing discussion 

For purposes of illustration we shall concern ourselves with 
tho three hypothetical question^ on student adjustment to col¬ 
lege life. We shall assume that each question has three cate¬ 
gories, which we have assigned them on the questionnaire, 

(1) How satisfied are you with college life? 

(a) Very satisfied 

, (b) Neither satisfied nor dissatisfied 

(c) Very dissatisfied 

(2) Do you think that what you are learning is worth while 

or not? 

(a) It la worth while 

(b) Undecided 

(c) It is not worth while 

(3) What sort of a timB are you having at college? 

(a) lam having a good time ' 

(b) lam having a fair time 

(c) I am having a miserable time 
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First, then, we must see what the conditions are under which 
each item can be said to be a simple function of scores derived 
from the distribution of the items To do this, suppose the 
responses to our three questions gave us the following marginal 
frequencies: 

Cettgoiy 

eke JVlW 
1 35 20 55 100 

Item 2. 20 60 JO 100 

2, 40 30 JO 100 

We can express these marginal frequencies in bat chart 
form (Fig. 1). 

i 2 a 



For the items to form a scale, that la, to be simple functions 
of scores derived from the hems, all those respondents who 
make the positive response (a) to item 2 must also make posi¬ 
tive responses to items 1 and 3, aa Guttman has shown (ibid), 
Thu group of respondents would constitute a scale type In 
like manner, the other scale types pan be read from the bar 
chart (provided, the Items form a scale) simply by connecting 
the ban with dash lines (Fig. 2). 
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It ii clear then, that, given the marginal frequencies to a 
■at of items and the relative position of the categories within 
each item, It is possible to ascertain which response combina¬ 
tions, from among all those possiblt for the items, constitute the 
ideal Kale types for those marginal frequencies. 1 But we still 
do not know whether or not the items actually scale We are 
laced with the problem of determining how close to the ideal 


Seels 

l 2 3 Frequency Typi 

20 1 

6 2 

16 9 

6 4 

26 5 


10 S 

20 7 


Figures 



Rttpents 

Combination 


c-b-a 


b-b-b 


e-b-b 


e-e-e 


tho obtained response combination* and their frequencies' 
actually come. 

If a tally of the response combinations obtained showed 
all to coincide with the ideal Kale type combinations and their 
frequencies as determined from the marginal frequencies, the 
nwponscs would form a perfect Kale and theTC would bo no 
further problem of procedure. In practice, however, such 
perfect conformity is almost never encountered. The response 


• It fa jmHjIb to diurmla* rt» mufeiun ranto of iali mxs thoiifh nor ihs 
aifaantmiu oI mpaiMa whUi cwituuto th*n by Uiing Gatauifi farmuk 


Muimu* namW U ieil« ij 

mba numbir nl hem (”Tb» U------ 

b>S An KxibjSc of t Seal*," MiMomplud Fiper, 1 
bma DnsJos, ASF, 1941) 


patterns obtained always vary to a greater or lesser degree f™, 
tho ideal Kale established for the marginal frequencies 

The problem, therefore, b to determine to what extent rh D 
response patterns obtained approximate the ideal Kale- to 

what extent the response patterns obtained may deviate L 
the ideal scale and still permit of analysis m icaie iernu anJ 
what tbs procedural implications are when the respond □«. 
terns deviate beyond such an cstahluhed point of tohrLe 
from the initial ideal Kale 

Gunman hss kc the point of tolerance for deviation at lSy 
In other words, at least 8S% of the total number of respond 
must fall within the scale'pattern, so that it u possible to repro. 
duce at least 85% correctly all the responses of all the rt*pen¬ 
dents from the scale scores. 

If there is more than 15% deviation from the initial ideal 
scale, we have evidence that there u more than one factor or 
variable involved This allows us two inferences, One u that 
the extraneous factors are of minor importance and mainly 
affect only the categories, in which case they can be panialled 
out by combining categories, The other Inference is that there 
are major factors affecting the items themselves, in which cue 
category combination will not partial them out and no scale 
can be established^ 

To illustrate what ta involved where the first inference 
obtains, let us suppose that a count of all the possible reaponie 
combinations for our three items on adjustment to college hie 
•hawed the following results (When large samples are in¬ 
volved, this can easily be done on the counting sorter mudiln* 
or with tabulating equipment if the data are put up on LB.M. 
punch cards. Lacking such equipment, one may conveniently 
determine the nature of die scale relationship by working with 
a sub-sample of 100 respondents.) 

Here wo have deviation from the ideal scale well beyond 
the 15% pant of tolerance. Our problem now is to esamine 
the nature of the deviation. 

A. glance at the ux response combinations which deviate 
from the idsal Kale types shows that four of them have one 
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<a2G»'«i-w 


thing In commoni the response (c) to item 2. If we consider 
categories (c) and (b) on item 2 to be equivalent responses 
from the standpoint of the eerie relationship of item 2 to die 
other items, then 15 of the deviant responses coincide with scale 
types: b-c-a is equivalent to the scale type b-b-a, b-o-b to 
b-b-b, and e-c-b to c-b-b Furthermore, tho reale types 
c-c-c and c-b-c ire equivalent and the sum of their obtained 
frequencies u equal to the sum of their ideal frequencies. 

Therefore, in order to reduce deviation from the idea! types 
and frequencies to lew foan 15%, it is possible to combine cate¬ 
gories of any item. Each combination elmloaces a discriminat¬ 
ing scale type, however, and a minimum of combination la 
therefore desirable.* It la to be noted that since a combination 
involves the assumption that two categories are equivalent in 
the scale, It is ordinarily possible to combine only categories 
which are adjacent to each other In the category hierarchy of 
an item. In our present example we can combine positive (a) 
with neutral (b) or neutral (bj with negatlte (c), but not 
positive (a) with negative (c) without including neutral (b), 
which would dimtime the item. From the Kale as having no 
discriminatory value. 

To facilitate catcgoiy combination it is generally desirable 
roret up i distribution table rudn as the one below (Fig 3). 
For purposes of illustration we shall show in the table all the 
possible combinations of response, which ones are the ideal scale 
types, and where the obtained frequencies fall Using our three 
items on adjustment to college life, we get the following' 

*U KWrferet vKh Gorenm’i fomuJi is foeocu 3. 
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Figure 3 


When we combine categories (b) and (c) on item 2, si wc 
hove already ascertained'to be desirable, our table looks as 
follows (Fig. 4): 
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In order to reduce the deviant frequency 10 for the response 
combination c-b&c-a (Fig. 4) we can combine either (b) and 
(c) on item 1 (in which cue b-bfcc-a coincides with e-b&e-a) 
or (s) and (b) on item 3. The latter will alio reduce foe 
deviation involved in the combination Thus, by the 

fewest possible combinations of category, we obtain the maxi¬ 
mum possible reduction m deviation and emerge with the fol¬ 
lowing rabla (Fig. 5) I 
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The »cile that emerge* from the data actually obtained for 
niMhiMitenucM now btpKiintaJ in buchut form (Fig. 6). 

Frequency Seale Reiponee 
I g 3 Ideal 8 Obtained Type Combination 
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Figure 6 

We can now aay that all thoee reipon dents who are included 
n ropontc (a) of Item 2 are alio included in (a) of item 1 and 
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(abb) of item 3, and as on down the reals. It le possible to 
reproduce the responses of any respondent from whatever set 
of scores one may wish to use to designate the five seals types. 

In other words, we can now discriminate five degrees of 
what we are calling adjustment to college life op the baits of 
our three items 

1 Those who a) are veiy satisfied with college life 

b) feel that what thqr are learning u worth 
while 

e) have a good or fair tune 
2. Those who a) are vary satisfied with college life 

b) feel that what they are learning u not 

worth while or are undecided 

c) have a good or fair time 

3 Those who a) are neither satisfied nor dissatisfied 

b) feel that what they a re learning is not 

worth while or are undecided 

c) have a good or fair tune 

i. Those who a) are very dissatisfied with college life 

b) feel that what they are learning is not 

worth While or are undecided 

c) have a good or fair tune 

5. Those who a) are veiy dissatisfied with college life 

b) feel that what they are learning Is not 

worth while or are undecided 

c) have a miserable time ( 

For purposes of illustration, all deviation was reducible in 
our example. In practice, however, complete reducibility of 
deviation Is rarely possible without combining all categories of 
all items, thereby resulting in no discrimination at all. 

When the amount of deviation is sufficiently reduced to 
make further combination of categories impractical, there is the 
problem of how to score the few remaining deviants (the devia¬ 
tion presumably having been reducible to 15% or leu). Sup¬ 
pose we had had such a response combination as a-a-c on our 
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three items by the time we had done all practical categoiy 
combination (Fig. 5). By scoring a-a-c with a-q-a&b we 
would have only one deviant response (c) on item 3. Any 
other scoring, as, for example, with c-bicc-c, would leave at 
Unit two deviant responses (a) on item 1 and (a) on item 2 
Maximum reproducibility of response from the scale score is, 
therefore, the principle one would ordinarily desire to follow 
in tbe scoring of deviants. (It occasionally happens that a 
deviant can be scored in several different ways and still permit 
of maximum reproducibility. The indeterminacy arises if the 
number of itoros is small and can be removed by using more 
items. Usually the indeterminacy is harmless, however, and 
it is sufficient to give such deviants the most neutral of the 
several best possible scale scores.) 

When many items and/or categoric* are involved, it u ob¬ 
viously too cumbersome to work with such tables as those 
above (Figs. 3, 4, 5), where all the fosnbls combinations of 
response are shown. It is not difficult, however, to work with 
rinular tables allowing space only for the response combinations 
actually obtained By omitting from the table all those re¬ 
sponse combinations which have a ze/o frequency, the resuldng 
abbreviated table is convenient. To illustrate, we could have 
W up the first table in our example with the three items on 
adjustment to college fife (Fig. 3) in the manner shown below 
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In the course of our discussion we have seen that it is pos¬ 
sible to determine from the marginal frequencies of a set of 
items what the scale types should theoretically be for those 
itqms and for the population responding to them. Then, by 
comparing these with the response combinations obtained and 
by combining categories, if necessary, we have observed how 
the existence of a scale can be established, how the remaining 
deviation can be determined, and what the principles are which 
govern the scoring of deviants 

In conclusion, it is important to remember an assumption 
which has made possible the procedure outlined Everything 
that we did involved the premise that the hierarchy of categoiy 
within each item was reasonably correct to begm^vith, even 
where we made combinations of categoiy If the content of 
our categories had been such that no assumptions could have 
been made as to char relative positive or negative value under 
each item, we could not have set up an initial ideal scale for the 
items, a step on which our entire procedure was dependent 

From the practical standpoint, this limitation is not too 
important for work in the field of attitude and opinion, since 
the order of categories within Items can usually be judged 
beforehand with sufficient accuracy. In some cases, however, 
it may be difficult to do this Any of the techniques for scale 
analysis can handle this problem of determining category order 
"blindly*, but it will ordinarily Involve far more labor than in 
the case (such as that outlined in this paper) where the cor¬ 
rectness of a pnon judgments is assumed. The tabulation 
technique of this paper is being extended to include ibis mast 
general case where the categoiy order is not known 


Figure 7 















A STUDY OF THE USE Or MECHANICAL APTITUDE 
TESTS IN THE SELECTION OF TRAINEES FOE. 
MECHANICAL OCCUPATIONS 

• j w McDaniel 

BibndtU Junior CoUsp 
AND 

WM.A REYNOLDS 
Saciuusu An Serna Gumnirul 

Wartime dislocation of the labor market m the direction 
of mechanical occupations emphasizes a need for jelective 
screening of trainees Postwar return to' trade, service, and 
professional occupations will demand still greeter attention to 
applicants for retraining. If schools are to retain control over 
their programs, they must provide for increased vocational 
guidance for youth just preparing to enter the labor market 
and Tot adults leaking occupational readjustment. Increased 
guidance demands improved methods for determining potential 
occupational fitness Training programs now in the secondary 
schools give the schools a fine opportunity for developing selec¬ 
tion and guidance procedures useful during wartime and Indis¬ 
pensable in vocational relocation after the war. 

The ideal test of job fitness is ^ tryout experience under job 
conditions and under expert observation. The practical limi¬ 
tations on the wide use of this selective device necessitates con¬ 
struction of substitutes Two general types of ouch subui- 
tutes have been devised One type of measuring instrument 
» a miniature tfork simple in which effort is made to duplicate 
the work situation under measurable conditions Another type 
of instrument attempts to analyze the measurable abstract ele¬ 
ments of the job activities, This type may reduce the situ¬ 
ation to elements that can be tapped m i language form or 
other visual pattern and results in pencil-and-paper tests of 
aptitude. A number of these tests are on the market, and some 
m 
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of them claim high correspondence with actual work succeit 
For most of them very limited data are available showing their 
relationship to tramability of potential workers. The gnat 
need is for specific local evidence of usefulness of such tests in 
selecting trainees for middle level occupations 

This paper reports a study begun in the spring senuat or of 
1942 at the Kem County Union High School and Bakersfield 
Junior College for the purpose of forming a teat battery to 
diet success in mechanical training courses The study *u 
based on the results of administering three tests purporting to 
measure mechanical aptitude to 206 students, of whom 1U 
were in high-school classes and 94 in National Defense Train¬ 
ing programs, Trainees in machine shop, aircraft engines, air¬ 
craft mechanics, wood shop, and welding wets tested on the 
Bennett Test of Mechanical Comprehension, the MacQucme 
Mechanical Ability Test, and the O'Rourke Test of M f chancel 
A bility, Junior Grade , Form C. • 

The criterion of success used to validate the testa was the 
median rating on four traita given the trainee by the instruc¬ 
tor, After careful instructions tn the use of the mechanical 
aptltudo rating scale the uistiucton rated the trainees on. (1) 
aptitude for learning as evidenced by ability to underatand and 
apply instructions to the joh, (2) speed and accuracy as evi¬ 
denced by ability to acquire muscular and manipulative ikills, 
(3) workmanship as evidenced by quality and precision of 
work done, end (4) mtereat-entbuBiasm as evidenced by eager¬ 
ness in getting at the job and staying with it. Ten-point scales 
were constructed and behavior descriptions placed underneath 
various points on the scales. 

Results 

Of the 206 students, 197 were males and 19 were females 
The females and those who had incomplete test scores were 
eliminated, leaving 147 males in the validating group The 
means and,standard deviations of the various tests for thu 
group are presented in Table 1. The data on the high-school 
students and National Defense trainees are presented sepa¬ 
rately. Although the National Defense group is superior oo 
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the Bennett and O’Rourke tests, the critical redos between the 
two groups do not exceed 2.7/ and for the purposes of this 
study the two groups may be considered to be comparable. 

Tbo mtercoiTtlationa of the subtertB of the batteiy ait 
shown in Table 2 Since the standard error of an r of zero for 
147 caws is .082, it is seen that only forty-two of the uiter- 
correlationa are significantly greater than three times the stand¬ 
ard error of a chance correlation Howevdr, an attempt was 
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made to combine some of the subtests to that a significantly 
higher rauluple-correlatiOQ coefficient would be obtained. 

The first combination of subteau to be tried consisted of 
•ubtests B,Mj, JM, and 0,, a* these had higher coefficients with 
the criterion while at the same time having lower correlations 
with the other testa. The corrected 1 multiple correlation coeffi¬ 
cient was found to bo JS, and the combination was rejected as 
being too low in predictive power 

The second battery tried was tht combination of all sub- 
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tests, excluding the totala of the MacQparne* and O’Rourke 
tests. The corrected multiple-./? was .43 When the predicted 
criterion acores were correlated with the obtained entanon 
scores the correlation was found to be slightly, but not signifi¬ 
cantly, higher, or 45 4 Because of the labor in forecasting from 
ten tests end because the efficiency of prediction is but 107% 
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better than chance for this batteiy, a combination was sought 
which would contain fewer tests hut would give at high an 
efficiency of prediction. 

The third trial batteiy consisted of the seven teiti B, Mj, 
Mh M», M, and O*. The corrected mutripie-eondadoa 
coefficient was found to be .45, and when computed by corre¬ 
lating predicted with obtained criterion scores, 47. The effi- 
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TABLE 3 

- 
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(xncy <rf prediction for this battery was slightly better, being 
11754 better than chance, 

A fourth battery was tried, consisting of the five teats B, 
M Mu Mi, and Mi, and the corrected multiple-/? was 42, 
1^ correlation between the predicted and criterion scores 
Jurtred a slight drop to 41 The efficiency of prediction for 
rhu battery ws« thus only 8 79% better than chance. 

The coefficient* of multiple-correlation foi the four tnal 
battens* ai« summarized in Table 3. The efficiency of predic¬ 
ant foreach multiple-* u shown in the last nght-hand column 

Since the multiple-correlation coefficient of the third battery 
jgdKiiei that it is the beat of the obtained combinations, the 
nttdicted scores were correlated with the obtained criterion 
gptt for each separate das* of mechanical work. 4 The data 

ffoe summarized in Table 4 

Fran Table 4 it may be noted that the multiple-* varies 
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widely from claw to class The coefficient for the high-tehool 
Machine Shop clan, * «.60, is sufficiently high for forecasting 
purposes, while for the National Defense A/C Machine Shop 
class, * - - ,42, it is useless . An examination, of the means and 
standard deviations of the classes shows that one instructor, 
Ro, rated much more severely than the others, and the negative 
correlation Indicates a different standard of scoring in that 
class. It is possible that the results of the study would have 
been affected in,the direction of greater forecasting efficiency 
if the data for this class had been omitted 

Tho small number of cases in the dames, however, should 
be noted as a factor introducing unreliability in the results 
shown in Table 4, as strict comparability between the untrue- 
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ton* filings could not be obtained by transmuting them to a 
scale with a common mean and sigma Perhaps more impor¬ 
tant than this is the fact that the ratings were taken to the 
nearest whole number on a ten-point scale, and were not suffi¬ 
ciently fine for meaningful transmutation. 

Further evidence for the validity of the forecasting equa¬ 
tion was obtained by predicting the scores of the nineteen 
women not included in the validating group. Fourteen of them 
were in a dan on Aircraft Engines, under an instructor not 
luted in Table 4, and five women were in the National Defense 
A/C Machine Shop class listed in Table 4. The multiple-* 
for this group was .78 The coefficient is slightly greater than 
three times the standard error of an r of zero for 19 cases. 

The regression coefficients (Beta weights) for the tests are 
given in Table 5. ' 
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It It evident from the relative magnitudes of the Beta 
weights chat the MacQuame Tapping and Dotting Teats, and 
the O’Rourke General Information Teat contribute the moat to 
dio forecasting efficiency of the battery 

Summary and Conclusions 

1 Three standardized testa of mechanical aptitude were 
administered to 147 high-school students and National Defense 
minces in five mechanical training courses, 

2. From the three major teats a combination of seven sub- 
tests was selected which had a multiple-correlation coefficient 
with the criterion of instructors* ratings of .47 It u probable 
that this value would have been raised ( If two criterion mea¬ 
sures had been obtained and an estimate of the true correlation 
of the battery with the criterion had been made. 

3 The forecasting efficiency of the battery differed for the 
five training courses, but its over-all value ia significant in show¬ 
ing that prediction formulae need not be confined to but one 
type of training. Evidence, while not conclusive, was obtained 
thit the formula would forecast a new group with an efficiency 
greater than if no correlation obtained. 

4 Weigh rings of the subtests of the battery indicate that 
die MacQuame Tapping and Dotting Tests, with the O’Rourke 
General Mechanical Information Test, contribute most to the 
forecasting efficiency of the battery. 
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A VALIDATING STUDY OF THE OBERLIN VOCA¬ 
TIONAL INTEREST INQUIRY 

L.D HARTSON 
CQiulla CoUisa 


In thB yean 1927-28 W H. Brentlinger and the present 
author devised a Vocational Inttrajt Inquiry for use in connec¬ 
tion with the counselling service which was at chat tune one of 
the responsibilities of the Department of Psychology at OberUn 
College. Inasmuch as a description of this Inquiry may be 
found in Fryer** Maaruranuiu of Intarasts (1), it will be un¬ 
necessary to reproduce it here. A statement may well be made, 
however, es to the hunch which motivated the construction of 
this particular form of inquiry. A rather detailed cl assification 
bad just been completed of the occupations in which the Ober- 
Un alumni were engaged (2) This experience suggested the 
desirability of a vocational inquiry from the results of which a 
man’s interests could be classified in terms of the general type 
of function which appealed to him most strongly. A hit of 
twenty functions was therefore formulated, quite a prion, and 
enough items selected to permit a paired companion between 
each function and each of the nineteen others. The Inquiry 
involves, therefore, 190 pairs oT comparisons and yields a score 
in each of the following twenty categories; 
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The principal use made of the Inquiry wu aa an aid in the 
counselling of student*. With a change w the administrative 
setup, made in 1928, with reference to vocational counselling, 
me o£ the Inquiry vu discontinued In order that some con¬ 
clusion might be reached aa to the validity of this form of voca¬ 
tional inquiry, a follow-up study was initiated in 1941 by E B 
Knauft and the author Copies of thejblank were mailed to 
178 men who were the original subjects. Responses were re¬ 
ceived from 121, or 68 per cent of the list It is unfortunate 
that the population ia not large enough to furnish more defini¬ 
tive information concerning the characteristic patterns of inter¬ 
est for contrasting vocational groups. The number is large 
enough, however, to furnub some suggestions as to the con¬ 
sistency of individual interest patterns over the period of four¬ 
teen years—time enough to permit most of the men to become 
established in their professions 

Valuation of the Inquiry 

1. Extent of Agreement between the Patterns of Interest 
Indicated by the Inquirui of 1927 and 1941, By ranking the 
function! on the basil of the frequency with which they were 
preferred, it u possible to compute rati difference a^relatlona 
between the two acre of choice*. For the total population of 
|2l the consistency of choices on the two occasions is indicated 
by an r of -717 =fc .077 (see Table 1)- The correlation! for the 
different vocational group* range between .899, for the 12 men 
m the field of aocial science, to .282 for five accountants; 
for nine Individuals who could not bo fitted into any group 
classification, from £42 to .063 The table also reports the 
correlations for three special aggregates For the nine artists 
and musician! in dua sample, consistency of interest is repre¬ 
sented by a correlation of -889, for the 27 college teachers, by 
an r of 804, and for the 30 men in business and industry, by an 

*fiif .668. 

2, Agreement between the 1927 and 1941 Rasponsei t at 
Indicated by the Most and Least Favored Functions, Table 2 
report* the three most favored and three leant interesting func¬ 
tion!, on the occasion of each of the Inquiries, for each sub¬ 


group This table shows that m the case 0 f those groups for 
which the correlation between the two response* showed a cor¬ 
relation m high as 726, at least one of the functions was aston* 
the three moat favored on both occasions, and With all but two 
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of the groups this is true of two or three of the function*. 
When one turns to the lease preferred functions, it is desirable 
to eliminate from consideration function XIII (Routine Actlvi- 
ties), since that w almost universally unpopular No other 
function is commonly eschewed, but at least one of the other 
two functions which was least interesting on the first occasion 


202 EDUCATIONAL AND PSYCHOLOGICAL MRA8UXBMHKT 



TUB OB ERL IN VOCATIONAL INTEREST INQUIRY 203 

is found at the bottom of the list in 1941 with all groups for 
whom the overall correlation* between the two series of choices 
i* at least .648, a* well as with three other vocational groups 
when the correlation is lower than chat figure 
* 3 Agreement of the Profiles mth Empirical Judgment as to 
the Functions Characterising the Different Vocations, Eiami- 
nation of Table 2 show* that the pattern of preferred functions 
shows marked contrasts when one vocation is compared with 
another, and that they conform almost without exception to 
common sense expectation Examination of the responses made 
in the 1941 Inquiry shows, t g., that the three function* chosen 
most frequently by the social scientists are* Diagnosis, Fornw- 
Utioti of Policies, and Research In response to the initial 
Inquiry, this group showed a strong interest, also, in Evaluation 
of Conduct On the other hand, they displayed least interest 
in Routine Activities, Deeds of Prowess, and Profetu'aiU] 
Entertainment The phyafbians show a distinctly different 
pattern Their initial preferences were for Diagnosis, Re¬ 
search, and Offering Service or Aid, hue by the time they 
had begun the practice of medicine, the function of Administra¬ 
tion Yd Advice had displaced Research The physical scientist* 
■bowed a similar preference for Research and Creative Activi¬ 
ties, but substituted Diagnosis for their initial choice of Practi¬ 
cal Construction Problems. The group of college teachers who 
were working in the general area of the languages report una¬ 
nimity in their first three choices on both occasion*; Creative 
Activities, Research, and Evaluation of Achievement or Prod¬ 
uct. The types of activity most attractive to the clergymen 
are* Oral Presentation of Reports, Evaluation of Conduct, and 
a,. either the Formulation of Policies or Arbitration (the set de¬ 
ment of disagreements). The final preferences of the lawyers 
differ widely from the undergraduate choices. They present * 
logical combination. Arbitration, the Administration of Advice, 
and Teaching. 

On the other hand, there are instances in which the pattern 
chosen seems unrelated to the occupation. This is notably die 
case with the men in the fiald'of secondary education, wheat 
more recent interests lie in the field of Cresdve Activity 
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Research, and Diagnosis (of difficulties) The teaching func¬ 
tion does not appear among their most favored activities 
Review of their vocational careers ahowa, however, that, in half 
J the cases, die teaching profession waa chosen only after 
another vocation had first been tried. Again, one would expect 
Financial Speculation (activities aimed at financial profit) to 
appear among tho most interesting functions of the business 
This appears to be true, however, only in the initial 
jMsJry. and then only with the advertising managers. These 
facts sugg®* 1 die desirability of an analysis of the Items which 
Kern chosen to represent those functions to see to what extent 
they are trojr representative. 

An Item Analysts 

In order to lay the groundwork for a possible revision of 
thef*fi*tfy» the individual items have been examined, Bret, for 
the purpose of eliminating those that are non-differentiating, 
and, second, for the purpose of selecting those representative of 
the functions having the greatest predictive value. The non- 
differentiating items were located by the process of computing 
proportional preference given to each item ui the 1941 
Jtqwy. Of the 380 items in the Inquiry, 184 were preferred 
ty 75 per cent or more of the subjects and were therefore elimi¬ 
nated Then, after the subjects had been grouped according 
io the vocation in which they were engaged m 1941, the relative 
popularity of the 196 items was computed for each vocational 
group. Examination of those proportions revealed 45 more 
items which were not distinctive of any vocation in which these 
particular men were engaged Their elimination reduced the 
total to 151 items One consequence of this procedure was the 
reduction of the number of items for some of the vocational 
categories to less than 16. The number of categories having 
ac least 16 items was six, the largest number of items for any 
category bang 37 The identifying names being used for there 
categories are 1) Commerce; 2) Merchandising, 3) Law and 
Social Science; 4) Medical and Social Work, 5) Physical Sci¬ 
ence, and 6) Literature, Because of the inequality in number 
of items under each category it became necessaiy to use pro- 
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portions rather than sumiriationa of items in making compari¬ 
sons of relative interest for the different activities, 

The next step consisted in ra-iconng the original responses 
on this selection of items to determine the extent to which the 
respondents' original answen would have predicted their final 
vocational choice At this stage the men engaged In Art and 
Music were eliminated because too few items won found which 
differentiated these vocations. This elimination reduced the 
test population to 112. (It should he noted that in order to 
re -a core on these 151 selected items it was necessary to cany 
104 pure of items—including 57 undifferential items.) When 
the original inventories were re-scared on there 151 items it wss 
found that 63, or 56,2 per cent of the men had the highest score 
in that category in winch it would be most logical to classify 
the work m which they were actually engaged in 194L 

Further examination of the items suggested the possible 
advantage of making a still more drastic selection of items. In 
order to equalize numbers it was decided to select just the ten 
items in each vocational category which were more highly dif¬ 
ferential For this part of the study use was also made of the 
responses made an the original Inquiry by two additional 
groups of subjects, 27 students in the Hamnia Divinity School, 
and 90 secretaries In city YMCA's m Ohio. The responses of 
these groups were studied with the possibility of finding items 
specific to the field of religious work. This quest proved, how¬ 
ever, to be fruitless 

With this final item selection, consisting of ten items for 
each of six categories, the Oberlin graduates who had responded 
to the second Inquiry were again tabulated on the basis of their 
first choice among the six vocational categories. (In addition 
to there 60 significant items it was necessary to retain 32 indif¬ 
ferent items, thus making a total of 92 items, or 46 pain,) 
Classification of these men under these six categories on the 
bans of their actual fine! vocation shows that 56.1 per cent of 
them engaged in fields in the area in which they made the high¬ 
est scores as seniors. 

The vocational history of 47 other men who had filled out 
the Inquiry as seniors, but who had not responded to our second 
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Inquiry, bang available, their original responses were scored 
with the 60-item form The results m the case of these men 
give a 66.7 predictive percentage, 32 of the 48 men having 
chosen on occupation in die one of die six areas for which they 
■cored highest in the Inquiry, 

As a final step ui the validating analysis, the 60 items of the 
abbreviated Inquiry were classified on the basis of the twenty 
functions which had been used in constructing the original In- 
ftury When there were parcelled out in terms of the patterns 
characterizing the interests expressed by the various vocational 
groups (see Table 2), the number of items for each vocation 
proved to be too small to yield significant differential scores. 

Summary and Conclusions 

1. A validating follow-up has been made of the Oberlin 
Vocational Interest Inquiry, after an interval of fourteen years, 
bared upon the responses of 68 per cent of the men tested in 
the year 1927-26, In an item analysis use was also made of 
responses from 90 secretaries of urban YMCA's and 27 students 
in a theological seminary, 

2. When the correlation Is computed between the rank order 
of interest expressed on the two occasions, 1927 and 1941, m the 
vocational functions sampled by the Inquiry, the coefficient for 
the entire group of 221 u 717 

3. Examination of the patterns of preferred functions shows 
marked contrasts between the different vocational groups 
represented, and agreement with common sense expectations 

4. Preliminary steps were taken toward a passible revision 
of the/flqnfry ui the form of item analyses aimed at eliminating 
non-differentiating items This process showed that, on the 
bans of the interest shown m the particular items used ift this 
Inquiry, there men fell into six categories which have been 
labelled as follows 1 1) Commerce, 2) Merchandising, 3) Law 
snd Social Science, 4) Medical and Social Work, 5) Physical 
Science, and 6) Literature 

5. The item study revealed 151 of the original 380 items 
which had been differentially selected by the members of these 
ria vocational groups When the original 1927 Inquiry was 


re-scored, using only there 151 items, S62 per cent of the men 
made highest proportional scores m the vocational area in which 
they subsequently chore their life work. 

6. The ten most highly differentiating items were selected 
for each of the six vocational categories, this constituting a 
further reduction of the total number of items to 60. Scores 
obtained from the original responses of the 121 men on this 
60-item Inquiry give first choice to that category which most 
closely corresponds to the final vocational choice in the care of 
56.1 per cent of them. 

7 Finally, the responses made m 1927 by 47 men who did 
not answer the second request were scored on the 6Q-itcm In¬ 
quiry, and with this group it was found chat, m two-thirds of 
the cases, the first choice was given that category represented 
by the vocation subsequently chosen, 

8. It u to be regretted that the small size and specialized 
character of the population involved In this study precludes the 
me of more refined statistical procedures for Item differenti¬ 
ation (3) The results, however, appear to harmonize with 
those obtained by assigning multiple regression weights to the 
items. They furnish further evidence, moreover, pf the possi¬ 
bility of predicting vocational choices of college seniors within 
areas which stress particular functional interests. 
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GUIDANCE SURVEY OF STUDENT PROBLEMS 

ELIZABETH GORDON ANDREWS 
Florida Stale College (or Women 

This uiveitigation was undertaken as a contnbution to a 
curriculum irmly being made by a sub-committee of tho 
Faculty Committee concerned with the functioning of person¬ 
nel work and guidance on the campus of the Florida State Col¬ 
lege for Women. The students who secured the data were 
graduate students in a class in Personnel Work in l941-'42 
These investigators visited as many of the girls as were 
found in their dormitory roams when called upon All students 
at the college reside in dormitories (or in approved Student 
houses). The dnnnltorics were divided between the two in¬ 
vestigators. If a student was not in her room at the time the 
visitor called she was not included in the study. A total of 
279 students, or 14 per cent of the total student body, were thus 
interviewed. i 

Each interviewee was presented with 1 a sheet entitled “Gui¬ 
dance Survey." On this sheet were indicated the purpose of the 
study, a suggestive list of problems which the student might 
have had since entrance to college, a hat of the possible sources 
of help that she might have consulted, and an example of the 
information desired. This study does not include the numerous 
times students are called to administrative and other offices. 
We limited our study to problems on which students volun¬ 
tarily sought assistance, 

Only those problems wore recorded which assumed some¬ 
what serious proportions For example, the investigator would 
say, “Have you had any health problems since you have been 
in college? By health problems, we do not man the times you 
may havo gone to the infirmaiy to have your throat swabbed, 
but instances in which your health was such as to definitely 
sos 
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**“ * mr * ch '“ l ’">*•“ D-mphiwry problem, mctaM 
only infraction of rate. but ilso interpretation rf ruip, 
curricular problems included the care cl „«d f or dronumf^' 
actmtiea at well aa need for .ncreanu, cl™ All 
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One ol tbc chief factnn m coining unreliability 
due to the uncertainty nf memory P 0 , emm,,) r J?" “ 
report homeeickneae at a problem more than upp cr -d!l„ 
However, thn. may be partly due to the .election tl„ t ’ 
and the elimination of the homeeiclc ae well .. ™* n 

students. Poorer 

Sources of Help 

Table l md.at« the frequency with which the various 
office, or individuals on the campus served the students mu' 

TABLE 1 

Fnqwtncy of Fo hrnuiy Jttgwm fa Aid 
Number uududai u nudy, 279 Snulfnt body, 2007, 
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Tathftl (other chin Faculty Adyfeua) 

SO 

1 

2 

99 

30 
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Viewed with regard to their problems 

Both uppareclreamm and freshmen report that they co„„l, 
the Regutrar*. office eolely regarding auch problem. „ 
credits, ichedula., cure and euch mortem a. pertain u ntZ 
era non 

The Business Office is sought by both groups chiefly y^tb 
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regard to financial mat ten related to college Recounts, Twice 
they reported seeking aid in personal problems but in both 
these cues the student had a personal friend in this office. 

The Dean of Student's Office functioned chiefly in three 
areas insofar as students voluntarily seek help from this office, 
extra-curricular problems, personal problems, and discipline. 

The four areas in which the Personnel Office was most fre¬ 
quently sought were educational, financial, vocational, and 
placement problems While personal problems were seldom 
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Urted rhie fs perhaps due to the objective approach to problems 
m tbit office which avoids labelling personal problems a* such. 
Placement is peculiarly a problem of upper-classmen, especially 
of seniors; hence it js w be expected that freshmen would not 
be concerned with this function of the Personnel Office. With 
this exception, upper-classmen and freshmen consulted the 
office mat frequently on the same problem areas, 

J rhe Infirmity was, of course, sought mostly in matters or 
th, with occasional related problems In which advice was 
sought from the physician or nunc, 

Th«Office of die Director of Residence was sought s limited 


number of times chiefly in regard to homeslcknau, roommate 
difficulty and occasional personal problems 

According to student reports. Social Directors am .ought 
infrequently, bu,t on a widely divergent list of problems includ¬ 
ing educational, religious, financial, hoalth, vocational, peruoal, 
and disciplinary problems. It seems that the type of service 
the Directors in dormitories render most to the students lies m 
the field of personal problems. 

Religious Secretaries are sought by students chiefly with 
regard to religious problems. The number of times the Secre¬ 
tary is sought seems to be much greater with upperclassmen 
t han w ith freshmen This may indicate that the Secretaries 
become increasingly helpful ae they come to know the students 
longer, or it may indicate that religious problems are more 
prevalent among upper-classmen than among freshmen. A 
previous etudy on thle campus would, however, indicate that 
the problems in this area are more prevalent in the earlier yean. 

The Student Counsellors ware seldom sought regarding 
financial, religious, health, and placement problems Thdr 
chief service includes assistance m educational, extra-curricular, 
and disciplinary problema 

Educational, personal, and vocational difficulties constitute 
the three outstanding problems about which Faculty Advisers 
are consulted. The small number of rimes Faculty Advuen 
were sought voluntarily was somewhat disturbing, but no 
doubt one cause of this is the difficulty of finding an hour at 
which the student and the Faculty Adviser have corresponding 
free time for conference. This again points to the need for 
Faculty Advisers to be given a reduced teaching load in order 
that they may devote more tune to counselling. 

It is evident from the student reports that Academic Deans 
assist students mainly with vocational and educational prob¬ 
lems. This does not indicate vocational guidance so much si 
counsel with regard to pre-professional courses, such at pft- r 
medics) courses, courses for nurses, teachers, etc, 1 

Freshmen consulted the Heads of Departments only qu ,f 
educational problems while upper-classmen conferred 
them on problems of finance, extra-curricular activities, ^ 
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monc, and vocational guidance. This is probably due to the 
fact that freshmen have very little occasion to become ac¬ 
quainted with Heads of Departments until they have chosen 
their major field, which frequently is not until the latter part 
of the sophomore year. 

Teachers other than Faculty Advisers make a valuable con¬ 
tribution to the guidance program by giving satisfactory help 
on educational and vocational problems. They are also often 
■ought by upper-classmen regarding placement problems. 
They may be sought m any or all fields of guidance, but are not 
called on frequently in problem areas other than education and 
the vocations. 

If the nine types of problems were ranked according to the 
frequency with which students reported that they sought help 
tat these problems, the results would be as givdn m Table 3. 

TABLE 3 

Naivrt of Proliant {» Wfuth Rrlf It Souf&l 



'Uu pcittnDgn Riven frier id ihn relit ion of the number of rimei help mi 
,u»|ii id ths nuhcittd typ« of praUsau by the deihpunri group la th« tout number 
af tael hdp «>■ uught on iA problem, liy the group la queiuon. 


From Table 3 it is apparent that the problems that trouble 
both groups most are the educational ones, and this is the field 
in which students feel the most pressing need for assistance 
Freshmen appear to need more help than upper-classmen with 
personal problems Perhaps this is due to the more obvious 


214 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 

sophistication of the upper-elasamen. Vocational problems 
rank, high with both groups, indicating that selection of voca¬ 
tion and preparation for a vocation are not confined to the first 
year but are also problems during the later years of college. 

Problems of discipline, religious and extra-curricular matters 
are not so great a source of disturbance apparently as are other 
problems. Perhaps students fail to recognize th&m as prob¬ 
lems and hence do not seek help with them. Students seek help 
most in educational, vocational, personal, financial, and health 
problems The attention of Counsellors might be directed to¬ 
ward improvement of assistance in those areas m which SUI¬ 


TABLE 4 

VdvntorQj South i by StvJnu {utw4>nt to neorii) fa Am owl OgUt 



1243 100 100 


dents report seeking help most frequently, and further investi¬ 
gation should be made to determine whether problems actually 
exist in other areas where students are unaware that they need 
counsel 1 

During the interviews, forty-eight students reported that 
they had had problems about which they had not sought help. 
Twenty of these cases were personal problems, the remainder 
being distributed over tU tho other fields It» apparent that 
personal problems constitute the oustanding area in which stu¬ 
dents realize that they need guidance but hesitate to ask for 
help 

Out of the sixteen possible sources of official assistance avail¬ 
able to students on the campus, this survey reveals that the six 
offices or individuals sought most frequently by students in¬ 
clude in order of decreasing frequency (1) Office of the Director 
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of Personnel, (2) Teachers other than Faculty Advisers, (3) 
Academic Deans, (4) Faculty Advisers, (5) Social Directors, 
and (6) the Infirmary. 

In Table 4 it will be noted that the actual problems in which 
help was sought by students during the period of time covered 
by the survey are very much in the same order as reported by 
the students, indicating that the reports were fairly accurate 
and thit the sampling by the method used was quite repre¬ 
sentative, The only change in order would be placing the Per¬ 
sonal problems below Vocational and Placement problems. As 
stated previously, this is probably due to the approach in the 
Personnel Office which seeks to minimize the “personal" prob¬ 
lem in favor of a more objective approach At the same time 
the confidential records of the office indicate the persona] prob¬ 
lem involved. 


INTERPRETATION OF THE KUDER PREFERENCE 
RECORD FOR COLLEGE STUDENTS OF 
HOME ECONOMICS 

BOTH T LEHMAN 
Th« (Kb Snu Uwrcnity 

For tho past several yean, die Kvder Preference Record 
has been used experimentally with freshman students, in con¬ 
nection with a rather extensive guidance program in the School 
of Home Economics of the Ohio State University. This experi¬ 
mentation has shown that the Record mikes a helpful contri¬ 
bution to the total picture of the individual which is needed by 
the counsellor, suggesting both vocational and avocational 
interests. 

However, interpretation of interest profiles has presented 
serious difficulties. Some hive assumed that, since the home 
economics curriculum has traditionally been built upon a bass 
of natural science, performance interests in that area should be 
strong; that, in fact, students who have little interest in scien¬ 
tific activities should he barred from entering or continuing in 
the field of home economics. Yet interest scores in science and 
marks received in science courses by home economics students 
have been found frequently to be in direct contradiction Simi¬ 
larly, one might expect to find a pattern of interests which is 
characteristic of home economists. Such has not proved to be 
the case. The weakness of the assumption is evident when one 
surveys ihp diverse fields which the graduate jn home eco¬ 
nomics may enter She may become a teacher, an extension 
worker, a hospital dietitian, a foods manager in a restaurant, 
or she may enter one of the numerous divisions of business or 
of social service. Obviously, it would be dangerous to base 
interpretation of an interest profile either on one's prejudices, 
or on a.gue» as to areas which are significant for specific occu¬ 
pations requiring a home economics background. 

nr 
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Such difficultios in interpreting the Record blocked its effec¬ 
tive utc with college freshmen in home economics. Therefore, 
the teat wu given experimentally to a sample of homo eco¬ 
nomics teachers, dietitians, and home economists who were 
employed by utility and household equipment companies 
Findings from this preliminary work indicated that there were 
real differences in interests of the three groups and encouraged 
an extension of the study. Other cases were added during the 
school year of 1943-44 

The Study Reported 

To date the Record has been checked by 239 home econo¬ 
mists. These are distributed as follows. 125 secondary-school 
teachers, 42 hospital dietitians and 72 home economics women. 
In business Of the teachers, 51 were volunteers from a summer 
institute held for Ohio vocational teachers} 74 others in the 
State were reached through meetings called by their city super¬ 
visors The hospital dietmons were secured through group 
meetings in several Ohio cities, and through a random sampling 
of the Ohio Dietetic Association. The business women were 
from -rarious parts of the United States, They represent a 
random sampling of the extensive membership list of the Home 
Economics in Business department of the American Home Eco¬ 
nomics Association, supplemented by a smaller number of Ohio 
women not included in this list. Of the business women, 10 
were engaged in restaurant or tea-room management, 11 m 
journalism, and 20 in foods promotion or experimental work, 
and 31 were employed by utility and household equipment com¬ 
panies All three groups of home economists represented a 
wide range—but ■ lairiy good balance—in terms of experience 
in their respective fields (Table I). 

Farm B of the Kvder Preference Record was used, except in 
the early part of the study when only Form A was available 
In order that all data might be used, the raw scores secured on 
Form A were changed to comparable scores an the later form 
by means of a transmutation table provided by the author of 
the test. Medians were found in each interest area and their 
perccmde equivalents located on p profile sheet. Mention 
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ihould be made alio or tho fact that only a pan of the torn 
group received acorn in the meehanicil and clerical aiau ZZ 
these areas were not included in the early form (Sec footnm. 
to the vanous tables ) 

While the Internet patterns of different persons as revealed 
by tho teat were highly individual, theca appeared to be quin 
dietinct differences among the three group, of home econombt, 
mduded in the .tody (Table 2). The reach™ rod d.etit,. M 
had high social service interests; the business women were 
somewhat lower. The dietitians wore outstanding m science 
interests; the other two groups were lower, although still above 
the average for college freshmen The dietitians were also 
TABLE 3 

Afedtn PcrrnlB* Stow o/ Home XcononltUj* ,A, nine Inttnit Ami of d, 
Koiet fnjtmus Rtcori m “* 
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much higher in the computational area than were the teachers 
and business women. Tho teachers surpassed the others in 
artistic interests, the business women, in the persuasive or gell¬ 
ing area 

In other words, the areas of highest interest for the teachers 
were artistic and social service, median percentile scores in those 
areas bung 71 and 83, respectively In fact, over three-fourths 
of the teachers (77 and 78 percent, respectively) had scores at 
or above the median for college freshmen in these areas. Thera 
was some evidence also of scientific interest, but this was not 
nearly so high ss in the case of the dietitians A breakdown of 
the teacher group In terms of years of experience shows that 
this pattern of activity preferences is consistent (Table 3). 

TABLE3 

Utile* PtrttmlOt Stout of Horn* Smumla Tteebri mb* CUuifiti et to 
Ymii riErntertm* 



<N»27) 

S-B 67 31 tf 3 68 20 34 83 U 

cm- in 

10-19 63 « 31 IS 67 37 22 85 39 

(N=J7) 

20- . 55 50 64 13 75 41 25 78 37 

CN=43) 


_ *S(iWlo iwdueial ind dtrkil area* writ nor »T»i]iU»far5l ef rh* tacbtii. 
TOmbre, m the 1-4 jr snwp/N-4; 5-8 yr*. 6; 10-Ip yu, 32, ind 20 jan sad 


An apparent gain in artistic interests and loss in the social ser¬ 
vice area in tho case or the teachers having the longest term of 
service may be revealing. 

The hospital dietitians, though fewer in number thsn the 
teacher*, also presented a clear pattern of interests Out it end¬ 
ing areas were computational, scientific, and social service 
Median percentile scores in these areas were 63, 82, and 80, 
respectively. Moreover, around three-fourths of the dietitians 
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had scores above the 49th percentile (69, 76, and 79 per cent 
in the respective areas). 

Interestingly enough, though the business women differed 
in certain respects from the teachers and dietitians, they showed 
no definite preference pattern Interests in as many as five or 
six areas appeared to be nmdar (Table 2), However, when 
this group was broken down in terms of occupation within the 
business field, differences did appear (Table 4) Certainty u 

TABU 4 
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to how real these differences are will depend upon evidence pro¬ 
vided by additional cases, for the numbers representing the dlf- . 
ferent occupations were small. Interests apparent in restaurant 
and tea-room managers were computational, scientific, and t 
artistic (71, 64, and 76 being the respective medians, and 70j t 
90, and 60 being the respective per cents above the 49th per- \ 
centile norm). Similarly, apparent interests of the women in J 
foods promotion and experimentation were m the scientific area < 
(median percentile 78, with '80 per cent above the 49th pw» 1 
centre norm), of women in home service and equipment, in 
jussive (selling), social service, and probably mechanical 1 
(medians of 68,68, and 62, with 68, 68, and 79 par ceiir 
cases, respectively, above 49). Interests of the journalists 
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centered in literary and artistic areas (medians of 71 and 78, 
In each case 82 per cent being above 49) * 

Use of the Findings in the Guidance of Collage Home 
Economics Students 


The foregoing findings suggest certain conclusions concern¬ 
ing the interpretation of interest profiles for homo economics 
students 

1 The study does not reveal a pattern of Interests which a 
characteristic of home economists as a group 

2 However, the student who is considering home aconomics 
as a field or who u already enrolled In it can profitably compare 
her own interests with the outstanding preferences of those 
actively engaged In various professions requiring a home eco¬ 
nomics background 

5 Such comparison should serve as the basis for a discus¬ 
sion of specific activities In the field being considered which call 
for certain interest patterns Far example, the scientific inter¬ 
ests of the hospital dietitian are clearly used m the planning of 
special diets based on exact scientific facts, her computational 
interests m the keeping of necessary accounts, the estimation 
qf diets of given composition, and the ordering of food supplies, 
SimDarly, the artistic and social service interests of the home 
economics teacher should be related to creative activities in 
home making and to a desire to work with others and help 
them. 

4, Ideally such discussion should eventually lead to explora¬ 
tory experiences on the part of the student to discover her abili¬ 
ties and whether she really enjoys such activities Perhaps this 
should follow particularly in those cases where test results and 


professional ambitions are in evident conflict. 

5. In any case, however, the interest profile should he used 
m relation to other factors,— the student's personality, intelli¬ 
gence, special aptitudes, handicaps, health, and educational and 
experiential background. Left to herself, the student is likely 

* Thraujhout th« unJy, pmatSa nomu hr Farm B u crabluhcd lor Ant 
jorraBcreaKii indnsiMi m utd, niha dun the more rmni norma far high* 
•dml flHi. T*m> latter ublo would atwn du horn* Monamati Hijhar la carunl 
duo they in rawded ill thb pip* bj machuiical tod literary famnatt, and kotw 
wfcil lowif bi total umtt mUmti, It would alto give cyan higfao- nadboa la 
me ireal for Tar 


to rely too much on test scores Often she would like a test 
which would make decision on her part unnecessary. On the 
other hand, the inexperienced counselor with too great a respect 
for tests may use the profile in a dogmatic fashion or without 
supporting evidence* 

6. Furthermore, it is important for both student and ad¬ 
viser to interpret the Preference Record in terras of what it 
claims to measure, namely, activity interests, not ability, nor 
appreciation. Thu has been found to be doubly important to 
remember in the artistic, writing, and musical performance 
areas, where ability or appreciation are so readily assumed to be 
the interpretation. 

7- The counselor must keep in mind also that, although 
certain interest patterns appear in studies of given occupational 
groups, profiles Hre after all highly individual, reflecting both 
possible vocational and a vocational interests, and may change 
with experience and maturity 

Further Work Needed 

This account of the present investigation is in the nature of 
a progress report. Need for further study is evident In the 
fim place, the validity of the tentative findings for dietitians 
and home economists in business must be tested by increasing 
the number of cases Then, too, date from other related pro¬ 
fessions—such as home economics extension, and the textiles 
end clothing area of business—-should be added, if the findings 
are to be of greatest use to the counselor of home economics 
students 

Finally, certain general questions need to be studied, 
Among them are two which face every counselor who attempts 
to interpret students' interest score* One of these is the big 
question of how much student activity preferences change dur¬ 
ing the four yean in college and what are important facton in 
such change Tbo other is the basic question as to whether the 
pattern of interests revealed in a study of professional workers 
is closely related to experience m and knowledge of the job, or 
is more a reflection of the selective effects of the specific under¬ 
graduate curriculum which was followed. 


THE USE OF STANDARDIZED READING TESTS IN 
TEACHERS COLLEGES* 


ALFRED M. EWING 
Tail Wndcyui CaUeg* 


What is the most efficient way to uoe the result* of reading 
tests? Whatis the most usable test in the reading field? What 
disposition is made of reading test scores? In an attempt to 
answer these and similar questions concerning the use made 
of standardised reading tests in teachers colleges, dare were 
gathered by means of a mimeographed two-page twenty-seven- 
item questionnaire which was sent to 167 teachera colleges 
One hundred three returned the questionnaire, making the per¬ 
centage of returns, 61.7%, more than twice that of similar sur¬ 
vey! in colleges and universities No reading teats were given 
-in 17 of these institutions, or a program was just beginning. 
Thus the data here given are from 8 6 institutions answering 
questions concerning the use made of reading teats and the use 
and interpretation of the results. 

The 86 teachera colleges using reading tests reported the use 
of 27 tests. Four tests, the lotvo SUent Reading Test, the 
Cooperative English Test (Form C2, Reading Comprehen¬ 
sion) , the NeLson-Denny Reading Test, and the Entrance and 
Classification Test forTtachert Colleger Entrants. English Test, 
are used much more extensively than are any of the other 23 
testa. In fact the four tests make up nearly three-fourths of 
all the teats, their respective percentages being, 35.5%, 20.0%, 
10-3%, and 6 8% This limited use of a wide variety of testa 
tl accounted for by local uao of the test, use by its author, or 
some other specific reason and not by general considerations 
such as ease of scoring, popularity of the test, or number of 
■kills or abilities tested. 


i Thk article la baud on ■ *tu4r dona id partial fulfifimeot rf A* mulnaunt 
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Reasons for Selecting Tests 

Hie ability to diagnose reading difficulties is given as the 
main reason by 16% of the teachers colleges for selecting tbo 
reading teats which they administer to their students. Follow¬ 
ing closely is the fact that it “meats their needs," 14%, Sim¬ 
plicity of scoring, 10%; high reliability and validity, 9%, and 
recommendation or selection by authority, 9%, are other 
reasons frequently given by teacher* colleges for selecting spe¬ 
cific reading teats. Twenty-two other reasons for selecting 
reading teats are mentioned once or twice. 

practices m th* Techniques of Testing 
Four departments, psychology, testing, English, and per¬ 
sonnel and guidance, administer the reading ten in half of the 
teacher* colleges. In the other half reading testa are adminis¬ 
tered by ton other agencies. 

The faculty ranks fint as the group moat often scoring read¬ 
ing teats in teachers colleges. This includes such departments 
as English, psychology, etc. There seems to be, from the data 
obtained, no one department by which the scoring is generally 
done outside of the faculty and assistants. 

The personnel and guidance office ranks fine as a plan 
where reading test scores are filed The deans’ and registrars* 
offices rink next while die English department ranks fourth. 

The personnel and guidance office also ranks first as a place 
where the scores are available or given out* The adviser and 
dean rank second and third, respectively.’ In eight schools 
scores are not given out, while in many other schools more than 
one department gives out the test scores 

Fifty of the 86 schools give test results to the students. In 
three of the 50 schools the test scores are given only occasionally 
to the student and in sk other schools they are available to him 

The Use Made of Reading Test Scores 
The purpose of obtaining reeding test scores, as stated by 
40% of the teachera colleges, ia to find the students who need 
help in reading. Twenty-two per cent more say for diagnosis, 
while 16% give appraisal of the student’s reading ability aa the 
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ocorea. Very few bona fide low scores emit and they are unul]® 
caused by laclc of speed. 

In College D teat scores arc used as one of the bases for 
advising students relative tp the academic load they should 
cany. Instructors may consult records to determine probable 
student abilities. Deans and Advuera find the records vafo. 
abla in giving them an index of the students' probable success, 
and a limited number of superior students are granted a partis] 
tuition exemption. They prefer to select superior students 
rather than attempt to save (for teaching) those of inferior 
ability and attainments 

Aa an example of the use of reading tests m solving pw. 
eonnel problems, one of the teachers colleges makes the (ol. 
lowing statement from a five-year survey in the college. The 
college records data from several tests including one on the 
measurement of ability in reading. From tbs great vatu- 
tion in reading ability discovered, certain questions arose. 
What curricula do these poor readers enter? How long do they 
remain in college? What funds of grades do they make? 

The findings were: Curricula —physical education, agricul¬ 
ture, home economics. No one chose art, biology, chcmuuy, 
English, music, or physics aa his major field. Years in col- 
lege —the mortality rate of poor readers is exceedingly great. 
Only 70% are in school at the end of the first year, 45% at the 
end of the second year, 20% at the end of the third year, and 
15% it the end of the fourth year. Grades —those chat remain 
make gradei comparable to gradea of other college students. 

Plant Used In the Remedial Reading Program 
Twenty of the 86 teachers colleges have specialized reading 
classes for improving the reading ability of the students; fifteen 1 
have both a specialized reading clan and a reading clinic, while 
eleven report only reading clinics Thirteen schools use special 
sections of certain courses, while nearly equally as many appir- 
J autly give no concern to reading. The clinic gnd special elutes, 
however, account for over 51% of the 86 schools whose atten¬ 
tion s drawn to actual improvement. 

Small schools, that is those below 500 enrollment, give 
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■lightly more eltentioi) to reading than do larger ichtiob, hut 

the difference ia floe ilgoificaot __ __ 

How some of the plans for improving reading actually opw 
■te it eeen from the eiamplei given, one each lor a Hiding 
drnlc, n laboratory, an individual enniemnee meihod, and m 
orientation course, . n™ 

From a cMdmg eltoie-One o! the Iraat gratifying tspett. 
ol tho program has been the oneertainty of remits. In aotne 
in.taneua railing dame, all day, conducting a conlenmoi with 
readier, after achool. and ipending carta town to conversation 
have produced no tangible evidence oi change »n the instruc¬ 
tional program. However, most teachere show a keen interest 
in the questions discussed. 

But tangible results are sometimes, oftentimes, almost over- 
wheJuufifr Witness the cue of Miss F, a college girl who made 
the following record from tho analysis of her eye movements as 
photographed by the ophthalmograph, 

■Md Je%.7 ftb.16 Meyli 

jS| | 8 

aJKSTwiI ’ ■ V ’ *4 

Awiaa fcnMaol talkmto kcmSi 


The increase of 94 words per minute can be readily appreci¬ 
ated when practically applied to the work of a student. That 
means 5640 more words per hour provided the rate can be main¬ 
tained, and assuming she reads live hours a day, five days a ^ 
week, her inereued reading schievement rises to an impressive 
141 p00 words per week, or oyer four books of 30,000 words 

A laboratory reports thit the word ^laboratory" was used 
became it implied a broader, mare general use than the word 
“clinic" and did not Umic rive function to diagnosis and correc¬ 
tion. TheroembereofAe daw foam how to setup and main¬ 
tain case histories, how to operate certain simple equipment, 
how «) administer, acme, and interpret many kinds of t«U{ 
and how tn locate and assemble this essential Information *nd 
data in practical situation*. Students leim to think of tests as 
aids and guide*,not as Infallible conclusions. 
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In a cue of individual help the director of reading improve¬ 
ment listed the lowest quarter of the freshman dies for spetis] 
help as indicated by results of the loan Reading Test , then 
interviewed the staff for freshman reading assignments ui col¬ 
lege subjects, resd the assignment himself, and made objective 
comprehension questions on the content. The students read 
the assignments silently id the presence of the director, they 
were told to read carefully but ware urged not to huny. Then 
they answered from what |hey remembered At the end of a 
time another foim of the reading teat was green. The improve¬ 
ment wi statistically significant. This type of work is good 
but it is not always possible to schedule a special class, unto 
eatra time waa required Tor they read the assignment! that they 
had to read anyway 

One teachere college used the orientation couree to improve 
reading ability under the following topical Ideas: Dixuinon W 
reading ae a tool, relationship of speed and comprehension; 
administration of timed infonnel teats—the students were re¬ 
quired to find specific answers to questions in their assignment; 
use of association tests, in which the student is given a word 
or two and writes all he knows about these words during o 
minute or two, development by students of informal tests, true 
and false, completion, etc, use of assignments to provide for 
individual differences by noting books that are easy or difficult; 
check of visual and auditory efficiency of students whose raw 
is low. 

Recommendations 

On the basis of the data obtained from the eoopM*MI 
teachere colleges, the following recommendations are made* 

1. More teachers colleges should stress rwnedial reading. 

Forty-one of the 86 schools said they felt their reading program 
was fairly satisfactory, 33 were dissatisfied, eight were unde¬ 
cided, three did not indicate, and one school said it ban no 
problem in reading. * . . 

2. Th?re is a definite need for both a good achievement^ 
in reading and a good diagnostic test, 

3. Remedial reading should have a classroom spptQW j y 
well ss a clinical approach for both speed and comprehereW- 
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, jf^ed improvement can be made without the uae of 
apparatus such as metronoscopea, tachistoscopes, 
(jT^vd corrective reading manualB, by giving a little time 
jTprjetfce, drawing attention to a few efficiency device* In 
p-mrieading, and using mimeographed materials 
5, Follow-up tests should be administered to determine the 
-mwoency of the results of remedial reading programs 
fiTffitb lew attention than one realizes reading can be 
apioved For example, the student’s attention can be drawn 
to ducardmg useless rnouonb, such as pointing, head or lip 
BOTemtnti, at whispcnng, to making fewer and shorter fixm- 
(jont to seeing phrases and sentences at a glance, to antict- 
pitmf what the author is going to say instead of passively 
ittempting to absorb his idoas, and to practicing reading at a 
4 Waiter than convenient" rate 


MEASUREMENT IN THE CONTINUOUS SELECTION 
AND COUNSELING OF STUDENTS IN A COL¬ 
LEGE OF PHYSICAL EDUCATION 
AND SOCIAL WORK 


J E. TODD* 

'nulnitiRia of Piper Gumatry, Appleton, Wbcsuu 

Springfield Colleoe u seeking to solve through research 
and measurement some of the problems confronting it in the 
selection and training of leaden of youth for positions as 
teachers, program directors, and directors of social agencies, 
such as the Y.M.C.A., Boys' Clubs, settlements, Boy Scouts, 
camps, parks, recreation centers, and other agencies of informal 
education Those in charge of the personnel services recognize 
two dear lines, of responsibility One is the function of select¬ 
ing and making sure t|iat only those who are competent are 
admitted and certified co the professional ranks. The other is 
the function of cqunscling the individual in order that he will 
find the right road for himself and make progress toward bis 
professional goal. 


1 Discovering Candidates (14) 


Two-thirds of the suitable candidates for admission are dis¬ 
covered by present students and alumni of the college who as 
physical directors, Y.M.CJL secretaries, high-school teachers, 
and social workers are in close association with high-school 
boys. Rather complete admission data are required—viz., 
high-echoal transcript, ratings by principal and others, scho¬ 
lastic aptitude test scores, medical history, and personal his¬ 
tory Each candidate is interviewed by at least one member 
of the Alumni Admissions Committee or the College Admis¬ 
sions Committee, persons who can judge the applicant aa a 


* Fonnariy Dimer td AdmUw and Sudani Bditfooa, Chairman of Aa 
FnduDu Yaw, SpnesWd CUlcs* I9J7-1ML 
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potential professional man Admission is bsaod on intellectual 
competence, pend rial and social adjustment, leaderihip ability, 
health, physical qualifications, finances, and interest in a pro¬ 
fession for which the college educates, 

2. Coptmuovi Counseling and Sikelioft 

The college provides fqr continuous counseling services 
through the college course beginning with the orientation pro¬ 
gram of "Freshman Days" (4), which includes getting settled 
in the dormitory, registration, testing, social affaire, and expla¬ 
nations of procedures and traditions! and which is earned out 
to a large extent by a committee of upper-cliumen 

The Common Freshman Year (13) was begun In 1937 to 
provide general education to meet the needs of students In 
adjusting thamselvn to the college community and life's re¬ 
sponsibilities in a career of youth leadership 

A series of psychological and educational test# of abilities, 
achievements, adjustments, and interests (1) and a senes of 
physical testa of organic and physical efficiency, motor ability, 
posture, and nutrition (7) form the basis of a thorough pro¬ 
gram of selection and counseling throughout the student's 
career m college. 

In the group guidance course, "Introduction to Education, 
carrying six semester hours credit for the year, Individual re¬ 
sult! of the tests are given to the students and interpreted aa 
an aid to lelf-appraisal and self-direction Along with these are 
considered the student's adjustment to college life, the meaning 
and purpose of higher education, methods of study, efficient 
reading, and vocational and educational planning. In the 
winter term, consideration is given to health adjustments and 
personal hygiene; and m the spring term, to religious, ethical 
and sodo-vocational adjustments. Against all these, the stu¬ 
dent checks hia own qualification! and maps out his curricular 
plana forth* note year and the remainder of his college course. 
He makes application for admission to the professional division 
of his choice and all admissions and freshraan-ydar records of 
the student, including a profile of *U aptitude, achievement, 
personality, and physical testa, accompany the application to 
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the professional division with the recommendation of the stu¬ 
dent's advisor and the Chairman of the Freshman year, Each 
freshman is interviewed by the Divisional Admissions Com¬ 
mittee and is accepted or counseled as to dropping out of college 
or transferring to another more appropriate college. Thus the 
orientation course, the testing program, and counseling not 
only form the bases of sound guidance to the student, but alio 
give many opportunities for objective appraisal of the student 
and careful selection for the professional divisions. 

Students enter the professional divisions at the beginning of 
the Sophomore year, Each man i« assigned to a special coun¬ 
selor who helpa him plan hie program for the next three years 
and supervises his work during the yesr. All men csrry a full 
program of Liberal Arts and Science courses selected to serve 
preprofessional values. Continued development In recreational 
skills ii required At the end of the year, students participate 
in the College Sophomore Testing Program. The Sophomore 
year is a transition from general to professional education. 
Advisors and divisional officers using grades, test scores, and 
ratings, appraise carefully each man and make decisions re¬ 
garding him The promising men are encouraged to eontiniie, 
and the doubtful men are advised to plan other channels of 
endeavor 

The individual counseling by faculty members of the pro¬ 
fessional divisions continues ui the Junior year The course of 
■tudy includes several professional courses, said two or three 
advanced couraes in liberal subjects are required. Supervised 
field work begins In the local schools and aoeial agencies, which 
gives practical experience and testa the^profcstional aptitude 
and interests of the student In the Senior year, counseling by 
divisional advisors, usually by the professional divisional direc¬ 
tor# themselves, continues. The proportion of professional 
■tudy Increases and the program becomes almost entirely pro¬ 
fessional Students can concentrate in their particular fields of 
Interest Field work increases and emphasizes experience in 
supervision and is more critically appraised. 

Placement plans by die fogr-year men ate made and creden¬ 
tials an filed in the college placement office. Correspondence 
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and interviews with employers are arranged The recommen¬ 
dation of men for particular jobi if the responsibility of the 
professional staff of each division. Final decisions and plana 
regarding the fifth-year program or other graduate study are 
made by those students who are qualified 

Those continuing for tho fifth year increase their profes¬ 
sional study and field work and complete their theses or re-, 
search projects if they are candidates for the Master's Degree, 
The same general program continues on this level with even 
closer faculty-student relatione in all professional and educa¬ 
tional mature. 

Every student n expected to spend one or more summers 
in work experiences of educational significance. Most of the 
students become camp counselors, camp directors, playground 
and beach supervisors, or serve on the regular staff of some 
social agency which docs not havo a special summer program 
Reports of these summer's experiences are required 

The intimate social relations and physical conditions of 
dormitory life are influential factors in the development of 
studenti Supervision of the dormitories is in the hands of 
a committee composed of faculty and students Upper-class 
counselors, chosen by the dormitory committee, counsel stu¬ 
dents concerning dormitory adjustments and personal ques¬ 
tions. Meetingi of these counselors are held regularly, and a 
program of educating the counselors to more effective service 
Is carried on, 

Thu* throughout the college program, continuous selection 
and counseling procedures a ns in effect and make use of psycho¬ 
logical and physical measurements, 

3. Descriptive and Predictive Value* of Tests and 
Measurements 

Measurement is useful in making decisions ui selection and 
counseling because of St* descriptive and predictive values 
The following pre-war data have been chosen as being more 
typical of the normal functioning of the college than data avail¬ 
able in the war years. The data in Table 1 give a picture of the 
average freshman student. These averages are the result of 
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Te*ts of agility, large and imsll muscle coordination, po»- 
ftire, and nutrition are also administered during the firat term. 
Analysis of data indicate* that the freshmen who expect to 
major in health and physical education are superior to those 
majoring in the other professional divisions in general motor 
ability and capacity, In general strength development, and in 
motor development. 

A factor analysis tif strength testa resulted in two com¬ 
ponents which can be * measured by a few simple tests, vifc, 
dynamic strength, which describes a composite motor ability 
criterion to the extent of 68%, and static dynamometrical 
strength, which was responsible for 24% of the reduction (8), , 

l wT* of 18 cardio-vaocular teat variables and 

the McCurdy-Larson Tut and the Tuttle Test isolated eight '• 
principal components, viz, three concerned W |tb pulse rates, l 
two With blood pressures, two with pressures referring to pos- 
Cl™) C ^ ,an ® 6 * # * nc ^ one With circulatory recovery to exerei^ 

Other factor analyse* of variable* in specific sports, ouch W '\y l 
■occer, basketball, and other skills have been made by E^ Vi 
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«ntmDT (6), Tibbetts (12), Humidcer (5), and othera. Im- 
theoretical and practical studies ot posture and aquatics 
5m bran mads by Cureton and of enduranca and nutmion by 

“Eft 61™ Mm " fflsa of **" P reiic “ Te "datums between 
t-t (cores and achievement, Measurement has been useful in 
piedieuns individual and group achievement 
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4. Effectiveness of Program 

The ultimate teet of the validity of the whole selection, 
counseling, and academic program u its effectiveness in pre¬ 
paring men for youth leadership Pertinent to this subject are 
the data on academic mortality, placement, and professional 
stability given m Table 3. Data on Springfield College stu¬ 
dents for Part 1 of this table covered 0 years, The initial place¬ 
ment data in Part 2 of the table covered approximately 900 
placements over 20 years The data on permanence of the field 
of initial placement were collected J> to 20 yean after the initial 
placements involved. Strictly speaking, of eourse, a rigorous 
test of the effect of the program would require conducting a 
controlled expepment of such magnitude as to he impractical. 
However, it seems reasonable to believe that the relatively low 
academic mortality and the high placement and success in 
social work may be attributed m part to the use of measure¬ 
ment and prediction procedures in selection and counseling 
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guidance through self-afpraisal 

FORREST E. HEWITT 
Horn MBhirr Set«J 

It ia Well recognised that some guidance programs fail 
because the records become so elaborate and cumbersome that 
no one uses them. In the guidance program at Howe Militaiy 
School, this difficulty has been overcome by arranging the ma¬ 
terial to that a student’s problem can bo ascertained with from 
three to five minutes of careful study. The cumulative record 
■yatem Is based upon the recognition of the following three 
problem fields. 

(1) Self-guidance 

■ If today’" students are to be prepared to ftdjuat themselves 
to this gadget-filled world of ours, it il moat necessary that they 
be encouraged to idf-guidance in the beginning of their high- 
sehool car«rs po that they can become mote self-reliant, can 
make better choice*, and will be better able ro meat new situ¬ 
ations wisely This awakening to the realization that one must 
be able to guide himself Is one. of the moat important single 
factors of «n individual's ea/ly experience. 

(2) Professional adjustment 

Too many high schools have allowed their senior* to leave 
school with the sole ambition of going to college If their parent* 
could afford to send them. Many of these youngatcra have, 
gone to college with no further aim chan to join certain fraterni¬ 
ties or sororities. Education again was incidental and the pro¬ 
fessional careen were too far removed for imitiediate considera¬ 
tion. Too often these graduates have found that they took the 
wrong courses and would probably have been happier if they 
had chosen other fields of interest. These people too often 
MB 
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Interpretation and analysis. Through the assistance of the 
counselor, each boy writes his oar* appraisal after drawing 
profiles of the various standardized tests , 

The first of the senes of tests u a check on their technical 
information in the ten major fields, namelyi human relation*, 
literature, government and history, biological science*, physical 
science*, machine shop, mathematics, business administration, 
office techniques, and architectural art On this same profile 
tho student's high-school training and occupational preferences 
an charted according to national norma for the academic level* 
Each b°y then csVea a senes of interest tests In the aame ten 
major Adds and compares these percentiles with hi* profile* in 
technical information, in occupational choice*, and high-school 
training. These profile* are then compared with the vocational 
plan* the student ha* outlined in hi* autobiography. Each boy 
write* ■ nimmaiy analysis of three profile* In the form of a letter 
ot reiay which u handed to the guidance counselor who inter¬ 
views each boy in regard to his interpretation These letter* are 
then given to the English Maiterwho check* them forg ram m a ti¬ 
ed erron and leucc-wnting fa cm. The corrected copie* are then 
returned to the counselor who send* them to the buys'parcnts 
with personal letter* of explanation suggesting discussions be¬ 
tween parents and ion* during the spring vacation concerning 
the personal and vocational implication* as shown by the 

profiles. 

At the first opportunity after his return from vacation, the 
Cadet add* to hi* cumulative record in the guidance office the 
additional information gained m hi* home discussion*. If it is 
, the boy’s senpr year, this inhumation is used os a basis for 
information in hie future life plan. If he plane to attend college 
the next year, this information becomes the basis for his Fresh¬ 
man college conferences and » placed in a folder and sene to the 
college of his choice. 

Previous to hu senior year, special aptitude teats *« given 
to the Cadet and a complete inventory accumulated during the 
yean he attend* Howe School This personal Inventory starts 
as non ai the student begins his formal life in the classrooms 
pqd the barracks, and accumulate* from the following source*. 
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become professional misfits and spend the re*t of their In 
wishing that they had chosen other career* 

(3) Self-appraisal 

Student! should and can be taught how to leem to , pp _i_ 
themselves, how to take advantage ol opportunities (orm-th. 
and bow to evaluate their progress. If a s.If-c* p l* nl Z; 
manual consisting ol appraisal profiles and cheek sheets u 
assembled and written for the student needs of each pattieultr 
school, the guidance program will not become lost m the daHv 
factual teaching. 3 

Tho source material for the Howe record system is recorded 
in profile form on a manila folder In which the cumulative 
material is filed This information is compiled from the follow¬ 
ing sources 

(a) Technical information and personal inventory tests 
taken by the student. 

(b) Reports from the Ma*ten, Tactical Officer*, the Head- 
master, and the Commandant, (These report* give a 
cross section of the work habits, conduct, and social 
adjustments of the boy.) 

(c) The Cadet's autobiography* 

(d) Counselor’s summary, 

Thu Hoto§ Profit System 

During the four day* of the anantation p&nod prior to the 
fall term, a complete check h made on each of the entering stu¬ 
dent’s achievement! In the fundamentals of reading, language, 
mathematic*, science, and the social studies. This buttery n 
scored end placed on Individual profile* showing the student's 
actual statue m the tool* of learning Thu information U amt 
to the Matters who begin their "special help" periods for the 
new student with specified difficulties 

The entire program center* around the building of profile 
showing achievements, interest*, personality, and social adjust¬ 
ment. At various times during the school year all of the boy* 
take a tenet of interest, personality, and special ability tests, 
the tcorea of which are placed on profile* by die Cadet* for later 
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(1) Profile* of the boy** daily scholastic work which are 
chocked by each Master under whom he Is taking 
courses. 

(2) Tactical Officers’ report* showing a picture of the boy's 
adjustments to life in the barrack* 

(3) Headmaster's profiles showing ratings of respect for 
authority, leadership, ability and willingness to co¬ 
operate. 

(4) Commandant’s report summarizing punishment* and 
reactions 

Sinee the rating on many of these factor* must of necesaliy 
be by subjective judgment, then profile* on the student’* work 
habits, emotional control, coopenPon, leadership qualities, and 
Future plan* are arrived at from various source*, thus giving 
a more reliable evaluation This summary profile in the form 
of R personal adjustment inventory ha* proved invaluable for 
counieling purposes and is placed in the Cadet** cumulative 
record file ui the guidance office. 

After the analysis of the profile* and inventories, then u no 
attempt to pigeon-hole the student Into a given profession w 
particular college. Instead, wise counseling Is combined with 
student freedom of choice The selection of profession* Is often 
born in the bi-monthly meetings of the College Preparatory 
Club For the under-classmen the broad choice of whether the 
boy want* to work with Ideas, thing*, or people very often 
comet from hit study of the career data and charts in the 
guidance office. This selection of a careqr by including the boy 
in the program from its inception gives the boy (he feeling of 
responsibility and security, as he knows that he ho? some defi¬ 
nite end specific plana for his future which can be changed, if 
necessary, to meet new situation*. 

Vs* of the Autobiography 

In order to help the high-school student to know himd(i 
the autobiographical method ha* preyed indispensable. 0t* 
senior entering the school for the first time lost fall, 
the dose of hi* autobiography, "I still don’t know exactly gMsSp 
I want to do, but writing this ha* helped me to know my*^; 
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better, and I think by the close of the Hemeator I will be able to 

ojtejn intelligent deasKm.’’ 

During his Junior year, another boy decided that he wanted 
« become a chemical engineer, but when he and the counselor 
Y'rillied b, s a putude and achievement profiles, he was made 
nwe of a definite weakness in mathematics (The author 
yjflvel that one of the most effective features in this technique 

that it offers the counselor the opportunity to assist the stu- 
Jat in discovering his own interests Bnd deficiencies.) 

When writing hia autobiography, this boy stated that his 
father was too busy to discuss his personal and vocational prob- 
lopwlth him. His counselor suggested that if the boy pre- 
uited a definite plan to his father, he was sure that hu busy 
father would take time to advise him This Junior not only 
*ttip a definite program, but also followed the counselor's aug- 
jationi to tiy out different jobs during Ins vacation. 

The broadening experience of this boy through the explora- 
wy jobs, and the new rapport developed between father and 
nn brought forth the proud statement from the boy when he 
gamed to Howe fur hra Senior year, "Now I know my dad, 
tad yrt decided that it would be better for me to change my 
erntr from engineering to dentistry.” He went on to explain 
revised autobiography his reasons for changing his coune, 
ad gave in detail hia new plans. 

Autobiographies are written during the Frbshman year and 
prised «ach consecutive year They are considered absolutely 
cttfidtntial and are filed with other cumulative material in 
cadi student's guidance folder in the counselor's office. Certain 
puts of the following autobiographical outline are revised and 
brought U P t0 date from time to time. 

I lifo history 1 

fa) Ancestry 

(b) Deicnpoon of homo life 

(c) What the boy thinks the parents expect of hin) 


II. Difficulties 


Home 

Social 

‘Physical 

Any ocher personal problems 
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1£I Achievements 

(a) Records, medalt, etc. 

(b i Sports 

jej Jobs held 

(d) Money earned and money saved 

IV Interests, hobbies, and activities 

fa 1 Description of interest in hobbles 

(b) Amusements preferred 
fc> Reading interests 
(d) Travel 

V, Emotional needs 

(O Personal and aaaal adjustment in school 

(b) Religious 

VI Life plans 

(* J Professional Interests 
(b J Vocations] experiences 

(c) Analysis of abilities in relationship to chosen geld of 

occupation 

___ „ (d) College and technical school plans (Seniors) 

VII Summary 

A short interpretation of the implications of the tat 
results m companion with the autuilognptij and future 
plana as previously mads with parents 

Each autobiography is read carefully by tho counselor, and 
if it u incomplete or not well written, the paper must be revised 
and rewritten. 

Practical Ovtconuu 

This type of cumulative record offers the counselor both 
tangible and objective facta Tor interviewing each student 
These records also help the student to decide upon more defi¬ 
nite future plans, and offer an incentive for him to do milch 
better school work. The profile system gtves needed motiva¬ 
tion for the superior as well as the average and poor students 
who have previously failed to work up to their abOity levels 
These objective records also bring to the students the impor¬ 
tance of personality development and its effect upon their social 
and school life. 

In meeting the needs of each student as an individual by 
teaching him how to study, how to choose and prepare for a 
vocation, how to get along with others, and how to interpret hu 
experiences which will enable him to solve present and future 
problems, the plan is helping the boya to adjust themselves to 
the rapidly changing world into which they are being graduated. 
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TRIAL AT YALE UNIVERSITY OF THE ARMED 
FORCES INSTITUTE GENERAL EDUCA¬ 
TIONAL DEVELOPMENT TESTS 

ALBERT B. CRAWFORD 
an 

PAUL B BURNHAM 
Yile Unnenity 

Among »vwal major problem, relative to demobilization 
with which our college, are Immediately confronted u that of 
evaluating, fairly and with reasonable assurance, the scholastic 
pmraifie of individuals now serving the Armed Force, Edu¬ 
cational provision, of the "G.I Bill" m due coune undoubtedly 
will bring to the colleges a flood of applicants for admission. 
Among these will be many "irregulars 1 '—those with broken or 
deferred educational histories In the formal sense, yet with valu¬ 
able training or new skills acquired In service Since these 
personal devalopmenta will be difficult, if not impossible, to 
assay in traditional coinage of the academic world ("units" 
and "credits"), colleges at last may be impelled to consider 
what the prospective student knows, and what educational 
aptitudes be can demonstrate, regardless of where or how these 
attributes have been obtained. 

The Armed forces Institute has made educational programs 
of extensive scope available throughout all major theatres of 
darning and operation Certificates 'attesting satisfactory 
completion of particular courses, or achievement-test scores in 
ipeejfic fields, are issued through "GJJ.Q-" for this purpose, at 
die University of Wisconsin, Madison, However, the Armed 
Forces Institute measure mbst useful to colleges in their selec¬ 
tion of future, demobilized students appears to be the General 
&4w*fional Development Battery of four tests, viz* 

MI 
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Test 1) Correctness and Effectiveness of Expression 
Test 2) Interpretation of Reading Materials in the 
Social Sciences 

Test 3) Interpretation of Reading Materials «the 
iiatwnl Sciences 

Test 4) Interpretation of Literary Materials, 


No further discussion of these tests is necessary here, since 
they have been widely publicized and are fully described id the 
Examiners* Manual issued by the American Council on Edu¬ 
cation, 1 This Manual contains valuable remarks concerning 
the nature and purposes of the General Educational Develop¬ 
ment Battery, and present! tentative cqllege norms for each 
test. It stresses the need for local norms as well, stating. 

These norms ire intended tn hdp the schools dead* what mini¬ 
mum test perfonnince should entitle returning wiMtivia pemms to 
a given amount of aademic credit in a given ares, Ben use of the 
heterogeneity just noted, however, there is danger chat certain schools, 
by unenoesf acceptance of the general norms here reported, may let 
standards incontinent with their local needs It u highly desirable, 
therefore, that each inidtudon, by idrniniicenng the avilian forms to 
its own dvfliao students establish its own local norms for use along 
with the genera] norms in the interpretation of scores of service nun 
and women reported to the institution by the United States Armed 
Forces Institute. 


To that end, particularly as it might affect the program of 
Yale Studies for Returning Service Men, an experiment was 
recently conducted at Yale University All members of the 
civilian freshman dais which had matriculated in July 1944 
ware Invited to participate; 135, or about one-third of the total, 
dfd ml Although the invitation put no pressure upon fresh¬ 
men—since only interested volunteers were desired—it stated: 
a) that parddpation would be of direct aemce to the Univer¬ 
sity and collaterally to men now in the Armed Forces, b) thsr 
reasonable compensation would be given for the time expended. 
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For each of these four test* comprising the A.F.I General Edur 
celtanal Development Battery* two hours are allowed. 

In appraising their performance, it id essential to note how 
well the tut group represented, in scholastic promise and 
achievement, the entire dess. Fortunately it was found that 
the random sample of participants in this experiment covered 
virtually the entire class range, from highest to lowest, in aca¬ 
demic standing, for the first complete term of Freshman Year. 
Average score of this group on the College Entrance Examina¬ 
tion Board's Scholastic Aptitude Test was 575 and on other 

TABLE l 

ATI Tail Group Comfani trflA Total FraiAman Clan of 1H7 


CEEB Vcttnl Bant* 
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• rranlaridM In icoxhluy-Kbooi weard otmaml KhofafU: pra&tMo* fa tom* 
0*0) uinaiMV lor mllary lertta, nirsni imtanin matrietthnt* jatunfl? 
muoa far thfl dtftnqa between 453 *wl 411 In toul daw numben rtpn*j*it «3 
■bnvc, . 

tests of the College Board or the Yale Educational Aptitude 
Battery of the same order—arnilnd 20 standard deviation 
above the dans mean Hence these voluntary participants 
ranked, in pre-matriculanon indices and later scholastic 
achievement somewhat above the median of Yalt freshmen*, 
i e, around percentile 60 rather than 50 in academic promise 
and accomplishment. Moreover, despite relatively high aver¬ 
age performance of this freshman group on the A F.I tuts, 
dispersion of their scores followed a normal-probability pattern 

. /The Battery amptowd h dur dbtvnsbli tUrouih the Coependve Tat Service, 
15 Amiunlini Avan*, New York Qty—l pm lid Iona n tbit actually u«d by the 


Ttme Allowances 

Msurawm time tUowed for etch test 11 two hour, i 
indicated in Table 2, many Yale student. £nuhed 
ably less time, The low correlation! there ihown 
that length o! time spent or etch tut bote Wet“. Sf 
to score made by these freshmen. The time 
generous and the tests (a. intended) thetetoie «em t0 g 
on 'power" rather than on speed ” ,w 
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Since ell four ten of thie hettety. Including that designated 
at Interpretation of Reading Materialt m the Natural Sci- 
enccj* are highly verbal, a considerable degree of positive inter- 
correlation among them would bo expected The Examntrt 
Manual rather surprisingly offers no data m this respect. The 
following table presents these, for the Yale freshman tut group. 
The correlations in this and subsequent tables were obtained 
from standard scores for the AFI teats, derived from raw score* 
by use of the convention tables furnished in the Exmmtr/ 
Manual. 

TABLES 

ImtrcomlahoRi MOdf AFI Tetit (Jf= IIS) 

TV II TFT 
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This correlation matrix indicates rather cleariy (on the 
basis of a small but representative Yale sample) that the 
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Armed Forces Institute GeruroJ Educational Development bat- 
tety is general, rather than specific or differential, throughout 
its several parts. We hasten to add that no derogation is im¬ 
plied by the foregoing statement. It is simply not in the na¬ 
ture of this Battery, nor In the aims of its distinguished pon- 
structon, to measure differential rather than general scholastic 
aptitudes; which indeed should he clear from its title and from 
objectives discussed in the Examiner? Manual Noverthclui, 
th«w coefficients are not so high as to suggest that all four tuts 
measure the same mental functions. It may be that similarity 
in form (i e., the faet that all four are highly verbal) accounts 
Tor the intercorrelations, and that dissimilarity in content 
accounts for their not being even higher 

Comparison of Armed Forcer Institute mtk College 
Entrance Examination Board Teste 
Although the content and objectives of A J.I, General Edu¬ 
cational Development Tests differ considerably from those of 
College Entrance Examination Board measures, certain ele¬ 
ments of the two serial] from Yale evidence, appear functionally 
to have much {a common, A chief difference of objective* 
concerns recency of formal schooling; Le, College Board cx- 
MTunatfcmn are meant to be taken in stride by students ntaring 
completion of their collcge-preparatoiy work and therefore to 
a‘substantial degree quite properly measure current, specific 
achievements. TheA.FI educational development 1 battery, 
n0 .. p V > P**ty» attempt to measure combined aptitude and 

achievement (however acquired) in retire general terms. Yet 
the following table indicates a rather high degree or corre- 
sjtoudence among those parts of each tret-senes which by tide 
seemingly represent analogous educational fields. The Yale 
experiment affords direct comparison in this respect of the two 
testing methods. Cwdjdares for admission to Yale are nor¬ 
mally required to tsktf the Cbilege Board Scholastic Aptitude 
Tart (both verbal and mathematical section*) and the Englvk 
Essay Examination, They distribute themselves rather widely 
WKwgdipijthw College Board options, according to subjects 
P* W in the aenior-high-echool ywr and Intended college 
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program. Consequently some of the correlations represented 
below, and others in Table 3, are based upon too small a num¬ 
ber of cases to warrant general conclusions By way of com¬ 
panion, first-term versus second-term grades in the Freshman 
Year (Class of 1945W) correlated: English 10, .61, History I<P, 
73, Chemistry 11 and 14, _S7; Physics 10, 76. 

Despite the numerical limitations of these data In some 
categories, it appears evident that the Aimed Forces Institute 
testa correlate surprisingly well with those of the College En¬ 
trance Examination Board In related areas—especially when 
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allowance is made for tho high average performance and there¬ 
fore somewhat reduced "spread" of Yale freshman on theA-FJ, 
Battery. > 

Relationship of Scores on AJFJ, General Educational Develop* 
ment Battery and Certain College Board Tests 
to Fvst-Term Grades at Yale 


A pragmatic and customary, though by no muni ideal, 
method of evaluating teat* id to correlate scores thereon with 
s ubseq uent grades in course. The latter, however carefully 
assigned, became of their subjective nature have dubious ire- 
tistiod reliability. However, m many situations thqy ofer' ^ 
the fairest criterion by which the relative value of differ^ ! 
prognostic measures may be compared. The nut exhjflft •» 



2 67 


268 BBUCATIONAL AMD PSYCHOLOGICAL MEASUSEMKNT 


— ii. S) nresuitfl ■ tenea of correlation. between ten acorea 
SfiUiJtn jr.de>, of pamelpant. in ehf. experiment, 
le b meopilteif that Kveral fartore boannj upon compin- 
. At College Entrance Examination Board teiti and 
5c i PI Road Educational DmlopmtM Ttiu in Tablea 
?f,df MyXt the maulta there given. All US men took 
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the nme A.FI tests but alLdjd not taka the tame College 
Bnnf tats, They all did take three of the latter, Scholastic 
Aptitude Test (Verbal), Mathematical Attitude and English 
Inaji. The remaining two College Board teats taken by each 
candidate for admission are elections determined by his secon¬ 
dary school course and prospective college major and are drawn 


from the following options: Social Studies, French, German, 
Spanish, or Latin Reading: Biology, Chemistry, or Pkynes; 
Spatial Relations (Three-Dimensional) Visualising Conse¬ 
quently the average on College Board Examinations is not 
based upon an identical battery For all men represented in this 
study, Each candidate's options, however, are appropriate to 
his expected freshman progranj, 

Most of the students represented took their College Board 
tests in Apnl, 1944, whereas the A FI. battery was admin¬ 
istered near the end of the first term. Consequently, the in¬ 
terval between taking College Board teats and obtaining the 
final grades for the term (lilt months) was considerably greater 
than that between A FI. tests and the and of term (one 
month) There is no way of estimating to what extent these 
differing time intervals may have affected correlations or the 
respective teste scores with the criteria. 

Again the number of cases upon which some of three con 
relations are based permits only tentative conclusions. It docs 
appear, however, that 

1) A.F.I. tests show promise of being wholly acceptable 
alternates for College Board examinations in the verbal sub¬ 
jects. 

2) For this sample group, A.FI Total Score correlated as 
well with Freibman first-term average in all counes, as did the 
average of all College Board tests, 

3) The A F.I General Educational Development Battery 
makes no pretense of measuring abilities in Mathematics, Me¬ 
chanical Drawing, or Descriptive Geometry. The College 
Board M.A T. and Spninri Relations tests are probably indis¬ 
pensable for prospective scientific or engineering major*. We 
have obtained no information regarding A.F.I. teats in specific 
branches of mathematics, but these would have pertinence 
chiefly for students who hava completed those' particular 
courses. 

4) Since College Board MMT scores are likewise some¬ 
what dependent upon previous msthematlcsl training, inter¬ 
ruption thereof might adversely affect individual performance 
on this test also. By its nature, the Spatial Relations score « 
lets likely to be so affected. 
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From joint inspection of the A.F I. test scores and freshman 
grade distributions, it was a relatively simple matter to de- 
tamune two critical-score levels Practically no students scon 
mg above the upper one had unsatisfactory first-term records, 
while most of those scoring below the other ranked well under 
tbs class avenge. As a result of this investigation it was voted 
by the Executive Committee of Yale Studies for Returning Ser¬ 
vice Men to admit candidates scoring above the higher critical 
level on the A F.I. Battery despite their deficiency in formal 
academic'credits, to reject or discourage (unless additional evi¬ 
dence more favorable to their chance* of success' is submitted) 
diode scoring below the lower critical level, and to examine by 
other means border-line cues falling between these upper and 
lower levels. 

For the reasons noted above in Conclusion (3) further tests 
(of the aptitude rather than formal achievement type) will he 
required of prospective engineering, mathematics or physical 
■gen re majors. 
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THE CRITERION 

HERBERT A. TOOPS 
The Qhb Sum Umvtnley 

Ih all tot work, and m malting prediction# generally, there 
must he—according to the current mode of the thought and 
Its consequent development into corresponding formulas—a 
unitary, general success wore, or criterion score, for each person 
or the experimental group by whose aid the tests are con¬ 
structed, combined, and validated 

Item analysis lor selecting the better items requires such a 
criterion. So does item alternative analysis, dune with tho 
purpose of altering, in the hope of improving, those confusion 
alternatives which are "out of line," which have, for example, 
positive item-alternative validity coefficients instead of the 
mudirto-berderirtd negative ones. For a multiple-choice test 
we den re such alternatives—for right answer and confusion 
alternatives—that the validity of each item may be as high as 
possible; or, to put it in another fashion, that m many as possi¬ 
ble or the experimental items shall have a validity coefficient 
above some arbitrary lower limit, say M for testa of intelligence 
at the university freshmen level In the former case, which 
presupposes the second having already been satisfactorily ac¬ 
complished, we are concerned with the picking of the individual 
liems such that, for the * item* chosen for a given test, the cor¬ 
relation of the Bum of the scores (the Rights score) on the 
items-as-a-whole (i-e., the test as finally constituted) and the 
criterion shall he a maximum IF the intercorrelations of the 
items are low, approximating zero, we may be content to ignore 
them; hut we can hardly proceed at all without validity coeffi¬ 
cients, or some reasonably adequate substitute therefor, the 
indices or which of necessity alio require a criterion for their 
computation. 

an 
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Having arrived at * "pretty good" items, each of deniable 
alternatives, we may hope, if the test is a time-Iinm ten m 
improve further the validity of each sub-teat by utilizing 
lug formulas, which, if we are to use them, "must duco**. 
emus” (ie, C must be negative in S-JJ + C IF). Thu too 
presupposes a entenon because the weight of Error*. (R) ^ 
Wrongs, (IF), i e., errors (S) plus omissions (0) relative* to 
Rights, can be ascertained (as vs. judged arbitrarily) only by 
the aid of a entenon and the multiple regression equation Thn 
selection of tests for a battery presupposes that cpnmdtnUa 
numbers of sub-tests are tned out against the criterion as a nr t 
of ultimate measuring rod, and that only "the few beat" lrc 
chosen 

, To weight these chosen tests in a multiple regression equa¬ 
tion then requires also a entenon The purpose hero is to so 
weight the several sub-tests that a maximum validity of the 
composite or "scale score" results Even if the weights are srbi- 
tranly chosen a entenon is needed in order to compute a 
validity coefficient of the thus arbitrarily weighted scale. 

Finally, then, we may say that we require a critenon for the 
general purpose of maximizing or optimalizing all sons of «la- 
uonships in the problem of prediction; for determining the poor 
alternatives and choosing the better items, for determining the 
best scoring formula, for selecting the better sub-tests, for de¬ 
ciding the weights to be ascribed to each in a battery or"soalo" 
for determining the validity coefficients of batteries or scales 
qf tests weighted either arbitrarily or by multiple regression 
equations, for determining the "most causal" of all the predic¬ 
tive variables (inclusive of traits and environmental facton or 
variables and social relationships) and determining tbeir rela¬ 
tive weights; and for determining what banc form of relation¬ 
ship (including therein "higher forms of the multiple regression 
formula") underlies the beat combination of all the "causal” 
variables For all such purposes a unitary entenon score for 
each person of our experimental population is indispensable. 

Yet success is not unitary! A concrete consideration of 
cases will emphasize the point, Miss A », let us say, +1 o in 
speed-of-typing, but is -1 o m accuracy-of-typing (that la, she 
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"is speedy but inaccurate"). Mias B Is -1 o in speed-of-typmg 
but is +1 o in accuracy-of-typmg (and popularly, "is slow but 
accurate"). If for each of these persons we add the scores, or 
average the scores, or (for these particular two persons) weight 
the scores equally, in oil three circumstances they turn out to 
be 0 a on the composite-of-speed-and-accuracy variable, or suc¬ 
cess-ae-a-typist variable; chat is to say, both are "mediocre." 
Yet by no reasonable stretch of the imagination ar« MUs A and 
Miss B equally "successful” typists The more true statement 
is to say chat they have a different kind, or a different type or 
a different pattern of success. Most employers of stenogra¬ 
phers will prefer for most purposes Miss B, because they hate 
errors, they hate the correction of mistakes and the complaints 
of customers and clients which inevitably result when errors are 
made. And besides, even if Miss B does less work, still very 
little of It has to be done oyer again. And doing work over 
rakes much time—much more than enough to do it correctly 
in the firet place—and costs money. 1 

Mne A and Mu* B do have a different profile of typing suc¬ 
cess. Thu suggests that, possibly, if we cnuld hold them to the 
samespeed of production (that is, keep Bpeed a constant) then 
their relative errors might measure their relative success, or 
conversely, if we could experimentally keep their errors a con¬ 
stant their resultant speed would yield us a unitary measure of 
their success; that is, that in either of the two alternatives we 
should have not two aspects of success but rather only one; 
and instead of having for each a two-traited profile we would 
have a unitary success score such as our ordinary regression 
and weighting equations demand. 

A hide thought will reveal tba fact that this u not possible, 
psychologically, for even these tuo variables; for hold down the 
speed of Miss A and instead of becoming more accurate (as 
almost everyone presupposes will be the result) the converse 
actually may occur; while speed up Mias B and her accuracy 
in all probability will improve; at least the normal expectation 


*A* irw, pojrnhrfjf, o«(Xm prodao it, pt it> undo jt («„ » inn It) 
mSK teifcil wtnk** w ramnw DC wlocauj |i enu) Nctd- 

lw> to wy, rfw fatryi in t»l U tgiul Boi tU» bmliijVa el ikt khhtm of the 


» rf mm ifmwn* flfbi 


•non m u iblwnd by texflvjau 


Is that, after a minimum of practice, such will occur. It all 
depends! 

Also, speed and accuracy are not the only variables m suc¬ 
cess. Success is not a two-traited variable compounded only 
of speed apd accuracy. If Miss A and Miss B are private secre¬ 
taries their employer considers as aspects of their mem or worth 
other abilities as well; how well, for example, they can spell, to 
cover up his ineptneas in this respect, the excellence of then 
grammar, to that they never asy in a letter "have went” even 
though that may be what he dictated into the dictating ma¬ 
chine; their ability and adroitness in answering the questions 
of visitors and in answering the telephone, and even their 
ability pleasantly and skillfully to entertain an influential cus¬ 
tomer until the boss's return In other words, even in simple 
jobs success is multi-dimensional Or, Anally, wa conclude that 
success on even a simple job is measured by the individual's 
profile on a multi-dimensional profile system composed of » 
critenon variables, whore, generally speaking, m ii greater 
than 1. 

The upshot of all the argument to date, then, would seem 
to be chat for an adequate treatment of the prognostic problem 
we should strive to predict the individual's success-profile, 
rather than to predict any unitary combination —however skill- 
folly combined —of those entenon variables. 

Thu prediction of a profile, n vt a unitary criterion store, 
may be done by the simple expedient* ofr 

1 Administering enough testa of a wide enough variety to 
predict well (a relative terml) each of the several subportwni 
of a entenon—the wage*, the foreman’s ratings of indispensa- 
bility, the accuracy and the quantity of production over com¬ 
parable time*, and so on 

2. Finding all the possible inter-correlations, 

(m+«) (m+n+1) 

2 

in number, of them critenon variables and the n test variable*. j 

3. Employing each of the m eub-cmenon variables in turn „ 
as a entenon and determining the appropriate regression eqil» \ 
tion for Its prediction by the aid of the n several tot variables'^ 
m each case. 
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. Making * different Y predictions for each examinee by 
of the m thus determined regression equations; and 
"Sduenthr plotting for him these 7 * predicted criterion scores 

ri* on « rosnlnr profils chirt, the plotted profile thru 

M ifce P«‘ fet “ ! pn ’ fil “ d " FT m S' iaC) °° „ 

Soth 1 profile, in common with ell profilee, reveels visually 
a relief in genera] three things, 

1 The general trend, or tenor, of the scores. 

i The strong points and the weak points of the individual's 
■ereral criterion "abilities " 

3 The dispersion of the criterion '‘traits’ about their cen¬ 
tral trend .... , . . - 

To the extent to which the several sub-portions of a cn- 
tenffli art highly related (possess a common G, or mathemati- 
a \ factor) will the standing in the several traits be more or 
fo, identical and tend, consequently, to result in a senes of 
ntinp Which are located in a horizontal lino The normated 
miner (the encircled "ovals") of all columns of the plotted 
profile would be identical if all sqch predicted sub-entem inter- 
tontlated perfectly. ’ . * . 

The /node of using such profiles—m selection, for example— 
h not generally agreed upon Consequently we are concerned 
•ore often than not with combining the several sub-criterion 
nibbles into a unitary criterion score, for in chat case the 
ample concept of a “critical score” applies. 

'Hio success, then, of an individual, about which wo prate 
10 glibly, is a complex thing, and if it is to be made, artificially, 
mm a unitary vanable must be compounded of the weighted 1 
mm of the several component parts, as one, simplest, concep¬ 
tion of the matter. If we accept that definition the problems 
beepine three in number* 

E** «h varlaUt, Y v wffl hire * »t»roUtd devfariao, Or,= 
JJ** , nVi, lod la therefore mmo conrtrkted than U or, Aaordbiplr 008 will 
uu! ■ End by tctul ami pure Don for nth at* * Y v»hie and djunnint die 
frpWrian of th*t variable Id order to find the d«jl# rang*, tha norm, for 
_ —l., kr the profile then of mhyno-1, deillng with aapect-2 d tbo aitsrgja 

i'll rfwuld be railed that if W* add acora. "wins no mtdtnlkr,” an really ire 
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1 To decide what variables are to be included in the cri¬ 
terion* i to decide what variables are to be weighted other 
than zero i 

2 To decide In what units—comparable measures—to 
record the individual's standing in the its several sub-variables, 
or component Variables, of the criterion. Unless the several 
sub-portions are recorded m comparable scores, the multipliers 
of the scores will weight the variables other than intended 

3. To determine how these are to be weighted, i.ru, to deter¬ 
mine by what specific weight* we shall multiply in turn the 
several sub-entenon scores of John Jones before adding the 
several resultant products to obtain his final “weighted” cri¬ 
terion score. Let us consider each of the three problems in 
turn. 

The Sub-Cftterion Success Variables 

Depending on the problem at hand one wdl have different 
success scores. Thus in a study of divorce, one would consider 
for inclusion in a criterion such variables as length of time that 
the marriage existed before the divorce, die happiness scores of 
the husband and wife at comparable times, or the sum Or aver¬ 
age of the two as an index or measure of the “happiness of the 
marriage,” the number (and the quality) of the children In 
which the marriage resulted before the divorce (if one is inter¬ 
ested in the qugenic&l aspects of divorce) and the like. The 
above statement suggests three corollaries. 

1 That the purpose of the study, whether legal, moral, 
social, eugenical and the like, will in part determine, if not the 
individual variables, at least the fields or realms in which we 
may look for the variables to be included. 

2. That there are always variables which are on the border¬ 
land of “test” and “criteria"; ones which in one study are on 
one side of the equality sign (of the criterion-weighting equa¬ 
tion) and in another are on the other Thus in a study of 
divorce m which the length of the marriage (stability or per¬ 
sistence of the marriage) u the focus of interest, the number af 
children logically Is thought of as being a “test,” the presump¬ 
tion (or hyppthesis to be tested) being that where there are 
children a rnamage probably is more stable or lasts longer than 
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where there are none. (There could be curvilinear relationship 
between number-of-children ( X ) and the persistence in years 
(T) of the marriage ) ' In another study where the center of 
interest is the social productiveness of the marriage, such as the 
lumber of children, the quality of the home, the wealth, happi¬ 
ness, inventions and personal adjustments secured, the number 
of children is certainly one of a nupiber of alternative "prod¬ 
ucts 1 ' or the marriages studied and quite dearly, then, is one 
of s number of alternative criterion sub-variables. 

3 That there u no universal agreement as to what consti¬ 
tutes “success” ui even one realm, not to speak of it as a gen¬ 
eralized measure in all realms of life. Concretely, we do not 
agree even as to “why, or for what traita of success we pay a 
given man one wage and another another." Obviously, then, 
to «y that "Mr. Jones is a successful man" means nothing defi- 
wu statistically, even if, by common agreement, it does mean 
that the man in question has been successful in a financial way, 
probably that he has secured promotions, recognition and even 
acclaim; and possibly, though not surely, that he has lived a 
good and useful life—m other words, it suggests much but says 
nothing definite and positive Since there are no universally 
agreed upon definitions of success nor even concurrence as to 
what vambles to include, we must be aibitraiy in any case, 
Wa may Seek to mitigate the possible bed effects of our own 
judgment by pooling, the judgments of other (competent) 
people of to tahai variables to include. , 

For our purposes here jt will be necessary to restrict the 
dmcucsion to some one concrete realm- The one of greatest 
development to date Is the vocational More criterion varia¬ 
bles hove been devised in this realm, probably, than in all 
others combined, Let us restrict our attention then to it. 

In this realm, then, we may ask ourselves what are the 
evidences of vocational success Lot uS enumerate «ome of 
those which have been employed In such studies, while in order 
to save apace we discuss each briefly at the point of mention 

1 Wages, In free competition, where workers are all 1 doing 

the same kind of work, and promotions “on the bjttii of merit” 
are promptly granted, wage* evidently are a fair tp good, If not 
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excellent, aspect of a man's worth. A little reflection, how¬ 
ever, will reveal that wages often are paid for quite other 
re tea ns than "merit displayed in free competition," One man 
is paid more than another of conceptually equal merit became 
he ie • relative of the boss; because he has more dependents and 
“Deeds it"; because he asks for raises more persistently; because 
the boes thinks well of him, or simply "likes him", because the 
trade union to whieh he belongs has a minimum wage scale for 
persons of his “experience"; because he has been employed 
longer, etc., etc. Conversely two men of conceptually unequal 
merit may be paid the same wage because the company pays 
only a few different "day rates”; or because promotions come 
only at stated intervals and wages generally are most highly 
out-of-lute Immediately preceding a "promotion period”, and 
the like. 

Again wages often nowadays are paid according to ■ "wigs 
formula" which "recognizes” many other factore besides boric 
"productivity” That la to eay a wage, or a part of a wage, 
may be a bonus intended os much to motivate die person along 
certain lines of desired behavior as to reword him for “merit 
already displayed “ Thus tardiness or absence from work may 
be penalized more heavily than the hours or nunutes lost would 
warrant In the hope of inculcating “punctuality” end "depend¬ 
ability" 

In general, equal wages will repeal equal ment only when 
tha attendant circumstances are equal, where, for example, the 
Dumber of days or hours is equal; and, Jo general! where tha 
u riskY‘ ore equal Where the wages are equal but the risk 
unequal, equal wage ia not an adequate measure of the relative 
worth of the two men receiving it, pmcretely, in the fine of 
two sales territories it may be harder to sail goods than in 
another (of different "need,” or demand, and socfa- cc pnomw 
.circumstance). Consequently equal commissions.received in 
the two mentioned would indicate greater “sales ability" on die 
part of the man in the “more difficult” territory, The problem 
of saying precisely how much better is not necessarily solved 
by an arbitrarily devised correction. , 



THB CMTBMOM 


2?9 


Absences (tom work, if not allowed for, make the ‘'pay 
envelope 11 on erroneous reflection of the individual’s ment.* 
Anything which arbitrarily reduces the standard deviation 
of the resultant sub-criterion Korea has a deleterious, or attenu¬ 
ating effect on the resulting Korea. Thus, in an effort to pre¬ 
vent "speed up” and consequent possible reduction of wages 
for dung a given amount of work, workara under bonus sys¬ 
tems often stereotype the output; that is to say, all or all but 
a few do on any one day practically the same amount of work 
despite their ability in individual cases—possibly in a majority 
of cases—to produce much more Although the record of a 
day, or any number of days individually may show this, the 
summated record of a week or of a month may not he revelatory 
of this situation Under such conditions obviowly the produc¬ 
tion u all but meaningless. Th« correlations with production 
of any truly good prognostic tests in this case will be attenu¬ 
ated, that la, greatly lowered Where uniform or practically 
uniform day wages are paid, soldiering on the job on the part 
of die more capable workers may have die same end result on 
any measuiw of quantity of production which may be collected. 

Where workers work In pairs, aiding each other in simul¬ 
taneous operations, the speed of the one will of necessity ha the 
speed of the other, as in all "assembly line" production and any 
criterion, other than “errors," becomes largely meaningless 
The respective wages in this case are a measure of the "wage 
formulas" employed and not of individual differences of ment. 

And one may be paid for “power to achieve" rather than 
for "achievement," because, for example, one can do hard jobs, 
or rare jobs or specialized jobs which another of the same labor 
force cannot. Even though this month both do the same work 
the fint of two men may be paid more because he, alone of the 
two, could do the unusual job if one came up, that is, the em¬ 
ployer is willing to, and does, pay for insurance that h« finiu 
“can handle all jobs that may coma its way." 

Often also there are notorious buses in wages. In Brasil 
it u said to be the custom for a worker to ask for, and- confi- 
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dently to expect, a raise with the advent of every baby in the 
family, and "ability" in this respect by no means is perfectly 
correlated with job-worth Again, women leathers, it is com¬ 
monly believed, often receive far less wages for comparable ser¬ 
vices rendered than men, the chewy, or justificatiun, possibly 
being that “men support families" while "women work for pm 
money" and should be paid accordingly. And of comae the old 
law of supply and demand still obtains! In such pares one may 
divide the investigation into two independent studies bared on 
sex" 


2. Production. Where all the workers of the experimental 
group are working at “exactly the same work," the quantity ol 
work produced is a good measure of the merit of the individual 
man Production in general» what occupations am for. Yet 
current production, even on excellently planned "bonus work " 
may reflect only, or largely, irrelevant factors such as a man’s 
temporary need for money, as for paying off a mortgage soon 
to come due, or reflect too largely his current condition as to 
sickness, fatigue, morale, or interest 

And, critical inquiry often meals that the conditio*! of 
work are not equal, that, for instance. 

One salesman has odd jobs to do which subtract from the 
tune in which he can give his full attennon to lus main job of 
selling. 

One salesgirl, as versus another, is required to sell a higher 
proportion of obsolete stock, which consequently sells more 
slowly than the up-to-the-minute merchandise. Or she may 
have alternative tasks which affect her sales effectiveness, as, 
for example, to holp keep stock In shape, to assist the buyer, 
and the like. 

One mao has a slightly higher gauge of metal so that bis 
"poundage" reflects this fact, rather than his "assiduity ” 

Another man's machine, “just like this one," runs at a 
slightly higher rate of speed than his neighbor's so that his 


poundage, and consequently hit pay envelope, reflect thu fact. 
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The “mils" of production, although assumed equal, may 
not «» fact bd equal In punching 80-column Hollerith cards, 
for example, one girl may punch daily 1000 cards and a second 
the same number. If the fint punches full cards (80 columns) 
while the second punches only 40 columns, the “column" mea¬ 
sure of production reveals the fint worker to be approximately 
twice “as good" as the second. 

Even with the units (columns, in the above example) com¬ 
parable the difficulty (“nslc”) of the tasks may be unequal so 
that 80JQ00 columns, or aggregate holes punched per day, of the 
one may not be equivalent to 80,000 columns of another. The 
one may be punching data requiring coding, which slows the 
uak, or may be punching from a rather illegible data medium, 
while the ether may have straight numerical punching from 
highly legible and exceedingly conveniently arranged data 
media, And oven if both are numerical the one may hive a 
data-mcdium where the entries are hard to locate (requiring 
many backward and .forward movements of the eyes) while the 
second hu a data-medium m which all the answers align in a 
column at the right so that punching them is all but an auto¬ 
matic job. 

Tune tends to relegate many factors such as those men¬ 
tioned to the status of chance, or compensating errors. Accord¬ 
ingly, the longer the time over which criteria are collected tht 
less important, generally but not necessarily, or# such "errors “ 

In some cates, also, one may so arrange it that there are ade¬ 
quate controls of such matters, that, for example, half of the 
subjects work for one week on the one project while the other 
half work on the other, and the following week the groups inter¬ 
change places, while some multiple of two weeks' wages is taken 
as the criterion variable. 

It would appear, (hen, that production Is likely to be a 
better criterion, if the attendant circumstances of its accumu¬ 
lation are under the control and supervision of the research 
s worker; if, m a word, he collects the data himself, rather than 
taking uncritically what may be handed t? him by a production 1 
officer or records dork. Only a very few firm* now collect 
routinely multiple, and reasonably comparable, aspects of the 
sticcew of their workere on va now variables. 
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3 Quality of Work The quality of work done by a 
“worker" is everywhere highly regarded as an important ab¬ 
stract virtue. In some cases, puck as die performance of the 
■tage, screen, opera, studio, prize-fight ring, and operating 
table, it is the paramount consideration Yet, like the fanner 
variables, it coo has its difficulties 

In bonus schemes bad quality of work frequently u dis¬ 
counted, and oftentimes more heavily perhaps than the “psy¬ 
chological" meats of the case would warrant. Such penalties 
often are “set high" as a means of discouraging their occurrence. 
Sometimes they are based on a quasi-rational principle as “the 
estimated time, often generous, which it would take to undo the 
incorrect or erroneous work and do it over again correctly or 
'errorlossly.*" 

Quality in most cases is based essentially on subjective 
human judgment. Limit gages sometimes give a highly objec¬ 
tive status to quality measurements where the precise size of a 
product w an important desideratum in the finished product 
repetitiously produced Evan here the man whose average 
error is .0002 usually receives no more wage, honor, ment, con¬ 
sideration, or even notice than he whose average error is 0005 
when J0Q1 u the allowable maximum deviation of the product 
from the ideal or perfect measurement One remedy for this is 
to devise measuring machines which reveal instantaneously, 
when a finished product is brought between the jaws of a mea¬ 
suring device and a constant pressure automatically is applied 
by means of a friction wheel, the amount of error on each piece, 
and its sign. This yields ■ distribution of the individual’s 
actual errors the mean of which may be token as hu "inaccu¬ 
racy," Such a distribution customarily v not routuw^ ctf-, 
leaud by industry It consequently ran be had only by special 
nquiry—involving the specially designed machineiy—intro¬ 
duced for that purpose 

An objective “rating Kale" may be constructed to measure 
quality. Thus by scaling cookies as to taste, and comparing the < 
taste of each experimentally produced cookie with the taste of 
scaled samples one may ascribe numerical magnitude! to the 
taste of samples of cookies, and so assign the ment of the several; 
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fnoki thereof, even though the scale be consumed in the 
.rtrtwl Sewing, soldering, cookie-cutter, wire-splicing and 
2^* scales have been used and, when properly need, 1 yield 
objectivity and probably also corresponding validity, in their 
Mjpective realms. r « « 

One may sample the products at periodic intervals, by 
-rtint for example, the errors made in successive intervals, 
collected at any time in hourly intervals by the inspector on hia 
wads in successive, frequent, but unanticipated intervals 
throughout the work day. This haa been called the tfmed 
sampling technique Errors in this may arise if the worker 
Unto, consciously or otherwise, to do a better quality of work 
[or a limited time when he auspects an inspection u “about 
doe.” 

The beat measures of quality result from those situations, 
ia jepetitive production, where all errors above an allowable 
minimum are automatically detected by objective inspections 
(eg-, by the electric eye). In this case the poundage of dis¬ 
carded work is a significant measure of the quality of work 
done. 

i,, j Breakage and if adage of raw materials or of partly finished 
poductB to which the worker is fitting additional parts are 
oportoot aspects of a man’s worth and so often are element* 
d a cntenon. In assembly or bench work often these consti¬ 
tute the core of the criterion To take account of different 
ipeeds of assembly of different workers one may take as his 
index of breakage or spoilage the percentage (or proportion) 
which the wasted material is of the total material processed. 

Clerks who otherwise are ideal employees may fail miserably 
m tfieir employer's eyes if they are highly susceptible to making 
Dan y clerical snore, illegibilities of writing, or errora in compu¬ 
tation, particularly the undetected ones which cost the em¬ 
ployer money or require him to adjust matters with irate 
customers. 

4 The Rate (or, alternatively, the amount) of Acquisition 
0 } New Skills. In aptitude tests, generally, the length of time 


taken by perrons, equally unskilled at the beginning, to achieve 
equal competence is a rather valid criterion measure. This pre¬ 
supposes “equal opportunity to learn ” It stresses the over¬ 
head aspects of employee worth In many jobs the training is 
costly. In war-time, for example, delays in production—due to 
training tune, or any other cause—are scanned more closely 
than usual Other things equal, chat employee who costs least 
to obtain, to tram and to maintain, is the most valuable. And 
if promotion » contemplated, the fast learners are those most 
worth retaining. There u at least a fur presumption chat they 
will be rapid learners on the subsequent job employing a good 
core of the same kinds of skills. 

This criterion may bo applied either to apprentices in a for¬ 
mal training school or to workers In training on the job itself. 
In fact, as will be shown below, it u sometimes possible to so 
arrange it that the Job itself becomes a prognostic teat with 
generally good all-round results. 

5. Supervisor's Judgments, These may be of "over-all pro¬ 
ficiency" and may be employed in lieu of a more satisfactory 
objective criterion; or they may be ratings, soy, of the several 
specialized ends or objectives of the work done, of which it is 
generally claimed, or claimed for the present, in lieu of objec¬ 
tive scales thereof, that only the foreman or other supervisory 
officer is an adequate judge. 

These often are notoriously unreliable in the technical, as 
vs. the popular, sense The judgment of a foreman in an eyelet 
department of a certain brass factory agreed with those of hie 
assistant as to the over-all merit of his men to the extent of only 
60, even though aided by a formal card-selectian-into-piles 
method of rating. He agreed only slightly better with himself 
on a subsequent occasion; that is to say the aalf-corralation was 
but little better. The antidote for such, of course, is repeated 
ratings, monthly or bi-weekly or even weekly ratings over a 
considerable period of time, with all the resultant ratings aggre¬ 
gated into a composite, automated or averaged, score 5y this 
expedient the composite rating may be made to approach as 
near 100 in reliability as we wish while its validity may be 
expected to obey the Brown-Spearman prophecy formula; that 


THE CB ITEM ON 


285 


286 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


il to say, may be expected also to increase somewhat but to 
approach rapidly, with increase in the number of ratings, a fixed 
ceding which generally is far short of 1.00 The crucial ques¬ 
tions here are: (a) just what “traits" Shall be so rated, and 
under what circumstances? and (b) how many racings should 
be secured, and from whom? 

The rating of the products of artists' work, pictures, etch¬ 
ings, and the like, are notoriously unreliable, intercorrelating 
no higher, perhaps, than JO. Even the reliability of the much- 
wanted medical skill and judgment in diagnosis is probably 
no higher than 50. In the latter case Brown's formula tells us 
that to secure a composite judgment with a reliability of .95— 
such as any group intelligence test, in its field, will yield on a 
2-hour examination or thereabouts—not to mention validity, 
would require the independent judgments of an assemblage of 
some nineteen equally competent doctors It follows that (a) 
often one cannot assemble enough “judges® who know well 
enough the subjects of the investigation to rate them accu¬ 
rately, and (b) that one often cannot, in many fields, obtain 
judgments or ratings “worth their salt." 

The reports of a training supervisor may bo of more worth 
than of a regular line officer Thus the worth of an assistant 
foreman, for purposes of rating job-worth of employees, may be 
(rearer chan that of hn superior, the fore man-in-charge, par¬ 
ticularly if the former supervises the men in their day-to-day 
work in the shop, while the latter concerns himself largely, or 
at least more than the former, with the general planning of the 
work and the paperwork of the department. In fact the latter, 
under this circumstance, may be all coo much influenced m his 
ratings by his knowledge Of the wages which they receive and 
, all top little by their more pertinent behaviors under varying 
Shop conditions The size of the pay envelopes or the exalted- 
uess of the titles of the supervisors docs not necessarily correlate 
highly, or even positively, with their “ability to judge." The 
correlations could be negative! 

If the foreman believes that “experience makes the man," 
ha ratings or rankings will be substantially a ranking of the 
men by age, or by experience And age, or experience, In this 


com will correlate more highly with his ratings than any other 
predictive variable Since age, or experience, in tfoat occupa¬ 
tions does not correlate highly with “competence” it follows 
that this attitude leads to spurious or attenuated ratings the 
badness of which will not be readily apparent. 

It if notoriously true that supervisors of teach era are not 
able to rate reliably—not to mention vflbdly—the teachers 
under their direction on even a fairly objective trait, 1 such xi 
the teacher's “ability to gain and hold the attention of the 
class " Consequently we must believe that any generalized 
objective such as the "teachers’ teaching effectiveness” cannot 
be reliably rated, even with the aid of formal rating scales, with¬ 
out many repeated visits for observation of the actual teaching. 
The ratings actually received often are a composite of the 
rumors about a teacher seasoned only with a dash of real knowl¬ 
edge thereof. 

The situation in school or industry in which more than 
two—or even two—persons know intimately all those under 
their direction is rut indeed Two ratings by different perrons 
are probably more valid than the same number, two, of re¬ 
peated racings by one judge, but if the former cannot be ob¬ 
tained the latter is about the only alternative * 

One thought here, however, where the number of judges 
oF necessity is inadequate, is to have fellow-workers, follow- 
apprentices, fellow-teachers, or fellow-pupils, say, rate one an¬ 
other. The extra number of raters may either partly or fully 
compensate for any assumed “lack of experience in (or compe¬ 
tence of) judgment." Such “judges" indeed are able, for exam¬ 
ple, to note certain aspects of competence not so readily de¬ 
tected by the supervisory officer. Workmen often knjnv, for 
example, when the shoddy work of a fellow-worker is being 
"buried” by an overdose of a beneficently-hiding coat of paint, 
like the doctor’s mistakes by the cafiketl 

One might even weight more heavily the judgments ren¬ 
dered by the adjudged better worker* than those of the poorer; 1 
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but If "everybody judges eveiybcdy"—-under conditions where 
nearly everybody knows everybody well-tt » a fair guess that 
the additional returns from such weighting generally may not 
pay for the additional time and trouble necessary 

6, Knowledge Achievement tests and trade tests of knowl¬ 
edge possessed may be employed as a criterion sub-variable 
There w more truth than fiction in the sanding joke that the 
eminent furseon'a bill or $200 for removing your appendix 
really is $25 for “removing your appendix" and $175 for bis 
“knowledge of what to do and what not to do” Popular 
opinion has it that there o an almost aero correlation between 
knowledge and skill Actually * moat trades the two correlate 

quite bighlyj w highly to fact qcJ tradfl l08ta (measuring 

knowledge primarily) are excellent teats for picking men in 
order of their ability u ta perform" on the job. 

> 7* Jab Tenure. This h comparable if the employees, say, 
OH entered the finn at tho same time and so have a good chance 
of being subjected to the same (historical) factors and condi¬ 
tions. Thu variable a only slightly related to "competence." 

I Improbably u highly related to that aspect of job success known 
, as *Uob satisfaction.* 1 It 'would not be comparable if buiinew 
cycle*, ware, or other "disturbances" did not "cotm” all tnv- 
' ployees’records equally. . , 

Following general reasoning that that profile »mo»t stable 
(statistically reliable)* and most representative of the person, 
other things equal, which is most unique (In the factor sense) 
it would Mem that it could be concluded that one should aim 
in predicting profiles to include as many uniquo variables as 
possible, while d on« is combining thwn into specific tub-ctv- 
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tuion Tariftblct, such u“»raur»cy," then the.anihlamdcfct 
ihouW be u “loedid’' with tht eub-erftenon "h*tot" (ha, 
''.ceuracy") u possible. In the eetoed cae, the eeeune. 
"teen" ehould be highly interarttletej tether then the eon. 
vereA- 

8 Supervisory and Leadership Ability The normal pros. 
re«s of a man m industry—and particularly to in war times— 
is to be given supervisory responsibility u fast as be demon, 
strates an ability to assume it, Where tho is the case the 
xuperwory status of a man is on important criterion element 
The foreman is distinguishable from the workman often, or 
even usually, not so much by "superior ability to do” u by 
superior knowledge, better Judgment, greater ability to solve 
difficult practical anti theoretical problems, ability to* lay out 
work, to arae-up repair jobs, to criticize faulty work, and to 
induce men cheerfully and uncomplainingly to work indus¬ 
triously 

The number of workers supervised Is probably of far less 
moment than the adequacy, or quality, of the leadership ren¬ 
dered This conceivably might be measured by some form of 
check lut. ' 

9. Incidental Factors. 

(a) Amount of supervision required. 

(b) Attitude toward supervision 

(c) Job satisfaction. 

(d) Adjustment to the work and to fellowworkers. 

(o) “Influence 0 on fellow-workers (e.g., morale-building 
influence). 

(f) Skills inventory, for example a measure of the “cor- 
rectnen of motions” the extent of possession of such a set of 
moiiona that present efficiency ia likely not only to persist but 
also to increase (improve with practice), Does this auto driver 
have the right "driving habits" as measured by a check lutf 
Does this salesgirl have the right sales tactics, as ascertained by 
a professional shopper? Does this profesiional swimmer have 
the “logically correct swimming movements"? 

Vitales points out that success is nor speed plus accuracy 
plui job tenure, but that it is an integrated whole. The diffi- 
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culty of explaining an o-dimenjronal whole in quantitative 
terms requires as the only alternative to combining the several 
■cores, ae by weighting, for example, that one treat the suceesa 
as a profile' 1 as above indicated- 

Bingham and Freyd have pointed out that objective criteria 
are more likely to correlate with objective tests, while rating* 
are more likely to correlate highly with tews of personality. 
If SO. ratings, often hitherto employed as criteria, »w doubt 
have perpetuated some W*is which ought to Have died a natu¬ 
ral death long ago. 

K criterion that U objective and uninfluenced by human 
judgment is usually “a more predictable measure" of job pro¬ 
ficiency than a rating based entirely upon lupervitori’ opinions 
of the workers' performance. 

A few general rulw applying more or lets to all euch cri¬ 
terion elements may not be out of place* 

1 It is Important to make provision for the orderly and syste¬ 
matic collecridfi of the criterion first, that is, ahead of adminis¬ 
tering tie tests’, otherwise the subjects may “get away" from 
one (by the route of graduation or dropouts of students, the 
transfer of workers, the shipment oveneat of soldiete, and the 
like), whereupon one will find that Ho has a fine bunch of ex¬ 
pensively collected and scored tests papers, but no criterion to 
tell him whether they are worrit anything or not. 

The test score* cannot of themselves tell one whether they 
are Of any worth. One of course must have a wide dispersion 
qf rest scores, but that of comae fa a fairly common character¬ 
istic of worthless tests. Thetas* must be fairiy difficult, ideally 
of such a difficulty that the average score is about half the pos¬ 
sible maximum scow, bat tjhxt alio is a characteristic of many 
worthless tests It must hwre wm.e reliability, but, in the ex¬ 
perimental stage* and other things equal, the reliability had 
betur be low rather than h^h to the end that its validity u 
1 *«ll 01 iu reliability may be greatly improved by lengthening 
it, Among wilting tests generally—and possibly even among 
■i tests pI a particular type as w«l^-th«« is no correlation of note 
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between reliability and validity Accordingly high reliability 
io itaelf warrant* nothing aa to the value of a test. And finally, 
even if a test ha* a large dispersion, tho right difficulty and « 
good to high reliability—all three—even that pattern of suppo¬ 
sitions merits nothing aa to the value of a tost. Validity mar 
be established only by a validity coefficient, in turn determina¬ 
ble only by the aid of a criterion. 

Much tune and care should be expended on the collection 
of adequate variables, Tho final test scale or prognostic battery 
probably will be no more valid than the yardstick by which it 
was constructed. And at beat it can be no more valid than the 
square root of lu reliability coefficient. Thu conclusion is 
achieved by manipulation of the formula for the attenuation 
coefficient. 

Possibly u much tune should be spent in devising the cfl- 
tenon as in constructing and perfecting the tests. This impor¬ 
tant part of a research seldom receives half the time or attention 
It requires or deserves If the criterion is slighted the tune 
■pent on the tests is, by so much, largely wasted. 

All criterion variables should be collected over comparable 
times The measurement of improvement, for example, m gen¬ 
eral should start at comparable periods in the several persons' 
individual learning or growth curves 

Criterion vonablee which accrue in time should represent 
the same amount of time sampling. It is better, Tor example 
to include jn s study all freshmen who persist to the end of the 
year, taking as a criterion score for each his end-of-the*fre*H- 
man year scholarship (to one definite point m time), dun ® 
include also cherem the averaged final success of those who 
dropped out with only one quarter' (one-third of a year) of 
college work and alto the final success of those additional oats 
who dropped out with only two quarters (two-thirds of *y **{).j 
of college work and so on The three populations mention™, 
have bver-til marks which have vastly different reliability x 
coefficients, say 70, ,824, and 875 (Brown's reliability fcrqmlt|, -J 
respectively, and considerably different validity 
This u partly in the interest of rendering comparable tit* 
ports, for example, the validity coefficients, regression 
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and the lit*. If one employe the maximally heteregene- 
HU group, the validity found by another who uses the same 
tun, otherwise comparable, would be exactly comparable only 
if he had th* same proportions of the three sub-populptions 

ff * I [f C djta 3 are missing in only a small portion of the cases and 
moply * few of the .criterion'variables these may be and per* 
faipi onhnanly should be supplied before beginning to eombuio 
tbe data Toopi" has developed a formula for this The for- 
omli usuries that for any missing score in a given variable one 
dunld supply the weighted average standard score of the per¬ 
iod jn question on those variables in which the data are avail- 
aUo, If the variables are highly mtercorrelated, and few scores 
ut musing, the assumption is justified. If a small per cent, 
preferably not over one per cent, of the persons cannot be rated, 
or the ratings (rating slips, for example) are lost, and there is 
reuon to believe that these are a random selection of the total, 
then the data-present coses may all be reduced to cencile ranks 
on the basis of the available Aft, while the miBBing-data cases 
may be assigned the rank SO. 11 To facilitate such “transmu- 
unon" on a large scale one may employ Hull's Table*. 11 

One may in some cases devise a mechanical device to count 
the juc«m of workers automatically, possibly even unknowns! 
to them. Thus a Veeder Counter may be so attached to a type- 
wnter or a Hollerith punch that It counts every key-stroke 
produced- If the one machine is employed daily and continu¬ 
ously by one operator, and one operator only, the total strokes 
of anyone day, week, or month, as the case may be, in equal to 
the close-of-business reading leas the reading-at-the-beginning- 
of-the-time-interval Over long periods of time oven errors, 
Mch as the use of the machine occasionally by an executive 
after hours to type a brief letter and the like, tend to iron out 
If such machines arc taken over by substitutes when the em¬ 
ployee in question is ill or absent this may introduce appreciable 

“Tdouj, Herbert A- "Tba Bdcrtzio ol Graduate Auaaon," PirtarMl 
hmJ VI (19U), 4HM71. 

uftartnaBy half by dunce ihodd bo uusneri tbs rank SO and bill tbs 

Clark. AfUimit Tuting, Appendix 1 Yenkara WadA Book Ctu, 
IBS. Pp- 491-tM 


292 EDUCATIONAL AND PSYCHOLOGICAL MBASUUMEMT 

errors into the resulting icorea, but less than would bo produced 
by absence and non-use of the machine by anyone if the days 
not worked are not allowed for 

Most systems of allowances for errors are not particularly 
psychological By means of addends the patterns of errors (or 
of correct responses) may be recorded 14 and later may be «ub- 
jected to alternative modes of analysis. To all intents and 
purposes the addend code number ir the original performance. 

Rate-setters and time-setters have many ridiculous customs 
and “allowances" as well as wise and wiser ones Their "stand¬ 
ard time," may be the ahartest of tough guesses rather than the 
result of a painstaking analysis They often are overly influ¬ 
enced by a "star (99 percentile) performance” rather than a 
statistically determined average performance. 

In cntenon-building we often imply the equality of numer¬ 
ous "dungs" which in fact never are equal’ 

1 That the motivation of the subjects Is equal. Thu is a 
particularly important consideration, 

2. That the "nik" is equal We here are confronted with 
such questions as. Do people ride cabs equally frequently In 
thu cab driver's "district" as In others? The distance traveled 
or the fares collected may not be an adequate measure if that 
is not the case If the houses are farther apart (as in the newly- 
built-up racoons of town) the potential nden per mQo are 
fewer but tho actual ride™ per mile may be more, 

Ie work available at all times so that this machine tender's 
wngei are free from error by reason of hia pay envelope not 
reflecting, or reflecting inadequately, "lost time"? 

3 That tho experience, or practice, or education,a equal 

4 That the human factor, as vs. machine factors, is the one 
which produces the basic variation in criterion ecorei of the 
various individuals of the criterion group 

5. That the persons employed for a criterion may allowably 
be combined in a society, rather than properly only be divided 
into two or more. We have mentioned sex above. The same 
can be paid for age, and in the case of machine-tenders possibly 
the age of their machines, 
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6. That the work environment ib equal Ventilation, noise, 
lighting and other conditions all have differential effects on 
individual workers which in some instances amount to biased 
qrora, or worse. Racial or religious antagonisms sometimes 
divide any industrial department into "good" and "pool" pto- 
ducera 

7 That the progress of work is not impeded by hindering 
factors, such as unavailability of raw materials, transportation, 
power or supplies over which the employoo has no control 

Comparable Scores 

Before the several portions of a criterion to be combined are 
Weighted, each person's X-score in each variable must be re¬ 
duced to comparable scores. For this purpose comparable 
■cores may be defined as any scores that have equal variability, 
Since ''success" at best is an arbitrary variable, one need not 
involve In the definition the requirement of equal moans in the 
several variables, If this is the definition adopted, then rank¬ 
ings (based In common on all N persons in each different cn- 
tenon variable, with "tied scares" outlawed) are comparable 

■cores; and so arc groan standard scores or Z’s {Zi + ^j, and 

also A.D. units, median deviation units, and the like. 

Because of their highly desirable algebraic properties, the 
ability to predict exactly standard errore of estimate, for exam¬ 
ple, if they are used, practice restricts comparable scores to 
standard scores or some variations thereof, such as Z’s, or 
r-rcorei, *kgd the like, all of which m common have equal 
standard deviations. Standard score*. *i ■ —. are most fre- 

quontly employed and have a property, sometimes valuable, 
that all the criterion variable^ automatically by Its use, are 
made to have equal means as well as equal standard deviations. 

The Arbitrary Weighting Formula 

The arbitrary weighting formula for combining the ik sev¬ 
eral Alb-portions of a criterion jnto a criterion variable, or the 
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several sub-criterion variables into a unitary entenon score, 
accordingly rat 

r,.^+M£ + .. (I, 

e, 0| a* * 

where X lt . . . , X m represent the wt several *ub-portions 
to be oumrauted, End Pu . . . , arc arbitrary weights, not 
precluding "logical" weights, to be accorded the re several por¬ 
tions, or variables, respectively 

If one is going to predict a profile instead of a unitary en¬ 
tenon score, one will naturally desire to secure as adequate an 
"accuracy score” or "speed score," for example—one of the 
basic variables of the profife—ai possible. It follows that 
possibly one will procure several different accuracy scores for 
each individual, eg., hit September accuracy scon, his October 
accuracy score, and hie December accuracy score- In this 
event, then, we ehall need to combine two or more scores (In 
die above case three) for each ‘Variable’ 1 (such a* speed, accu¬ 
racy, bonus earnings, etc.) available. Consequently in this 
case we sol] will have the problem of combining scores, end the 
above formula with corresponding adaptation still la appro¬ 
priate. 

Thus we cannot escape the “weighting" dilemma. Even if 
we have Only ons criterion score, we mil would weight the 
variable with & -1, in the bids system below, 

A marring &g*r of Sub-VaricbUs ‘ 

If one ts combining “accuracy" scores and "error” scores one 1 
may reverse the signs of the "error" scores so that they Wilt 
combine properly with the former This esn be done either 
by giving a negative sign to the (ft op to tho error scares, in the 
bids system below. One of the beat systems is to transqmre 
all the original scores, X t \ to transmuted score* on such * vari¬ 
able by rim formula, 

( 2 ) 

where fit Is the bids-tntportanc* of the trait, a positive reign}* 
tude, and Zg is taken of such a magnitude that all rtsulhOff 
’jTi'-scona are positive. It follow* that the resulting 1 

all b* positive but that the Ires meritorious, (eig, 4^®; 
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versa. 

The Buis System of Arbitrary Weights for the 
Portions of a Criterion 

The several arbitrary weight* flu P», .... 0, of formula 

(1) most conveniently can be obtained by the bids system pro¬ 
posed by X. L Kelley for weighting factore-of-ment in rating 
the SAT.G in World War L In essence it involve* the fol¬ 
lowing elements: 

a N judges, at least above a minimum of competence, 
ascribe to each of m entenon variables blds-of-relaU'/c-impor- 
tance, In napeet to their judged importance, under the con¬ 
trolling principle that the sum of the bids allotted shill exactly 
equal a certain predetermined quantity, say 100 The 
ation u thus rendered as close as may be to that of, "all other 
things equal, what per cent of the total importance (variance) 
can be ascribed to a deviation of It, in this variable of concern M 
It will be noted that this is one of the central points in our 
notions or variance. Consequently it is to be presumed that 
statisticians who have dealt with this specific problem, after an 
adequate on-rhe-job study of some weeks or months, may be 
more capablo of doing what is statistically demanded than th« 
nm-of-the-mino foreman or supervisor who Is about the only 
available other source of "expert judges " 

b, AFter the judges have rendered independently tholr Ver¬ 
dicts of bids on the several traits, preferably m integral num¬ 
bers adding to 100, one may weight all the judgments of a given 
judge. If desired, according to his assumed comPetence-at-}uig~ 
wg Thus ope could'welght the judgments as the square toot, 
or cube root, say, of the judge’s ysare of experience m super¬ 
vision up to some maximum—tay 20 years, If aevetal expert 
judges are available tho simpler procedure is to assume that all 
are equally competent, and accordingly by merely adding the 
stores one in a sense weights them equally . M Thus in ®vwy 
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case we obtain a weighted (summated) criterion score far each 
individual 

c. A compromise'set of bids u arrived at, possibly the 
rounded averages of the several judges' bids on a given trait, 
No harm is done, and some gain in interpretation is secured, if 
these are so rounded that their sum also is 100 

d The compromise bids thus secured are the values of ft, 
Pa, 0i, . . ,0. of formula (1). 

The judges should pay no attention to die signs of the p’s, 
as m the case of “accuracy” and ‘errors" above, but should 
judge the traits aB-if-they-werc-of-ebaaluta-aign, 

Points to be noted in the ascription of the bids 1 ' are* 

(1) Variables repeated in other variables (having a high 
correlation with other variables) should receive a low weight. 

(2) Variables which are subject to error, other things being 
equal, should receive lower weights than those not so subject 
to error 

(3) Variables representing an adequate sampling over a 
long period of time should receive a high weight, other things 
being equal, relative to a variable which if a sampling ol only 
a short performance of a similar trait. 

(4) The bide should be made up independently, ue,, with¬ 
out the judges conferring during the Original distribution of 
bids 

(5) After each of two or more judges independently has 
distributed 100 hide to the m traits, as much revision, before 
comparing remits, desired should be allowed. 

(6) After the bids are thus secured, a set of compromise 
bids should be made up. These likewise should add up to 100, 
In this construction of the compromise bid, the above fim three 
principles, and all other pertinent ones, should be invoked In 
the discussion, to insure that a fair set of bids result. It may 
be found, far instance, that two ideala daah; the use of assis- 
tantshipe, etc., ai rewards for the encouragement of good 
scholarship, and their use to obtain cheap labor for depart men- 
tal routine duties, for example. It may be found necessary to 

uTWkm, Htffctrt A. "Tha Selection of Qraduiu Anlmntt.” The Ftrnmmd 
Journal, Vl(I9Ji), 457-472. 
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make some distinction between these two; even to the extent 
of establishing two sets of weights, and evaluating each candi¬ 
date’s menu from the two points of view, successively It 
would seem to be good departmental policy, in that event, to 
evaluate for both appointments every candidate applying for 
either of the two types of appointments, irrespective of tho fact 
that the candidate applied for only the one appointment. 
Otherwise, from the department’s point of view a capable assist¬ 
ant, for example, may be lost if there are not enough scholar¬ 
ships and fellowships "to go round”, while from the student’s 
point of view, often he will find the alternative position offered 
him to be veiy acceptable. 


S 

* PERSONALITY AND INTEREST FACTORS IN 
DENTAL SCHOOL SUCCESS 

CLAUDE EDWARD THOMPSON 


Tee purpose of this study was to determine whether or not 
certain criteria of success in Dental School are significantly 
related to scores on personality and interest scales. Previous 
research by the writer (15) on the relationships between motor 
and mechanical abilities and success In Dental School required 
that he interview personally over one hundred practicing den¬ 
tists and the faculty of a College of Dentistry. These men 
were almost unanimous in claiming that personality and inter¬ 
est factors are of as much importance as aptitudes in determin¬ 
ing success m dentistry. 1 

Published reports of research and reviews of the status of 
selection and counseling techniques in dental schools (1, 2, 4, 
5,6,7,8,12,13,14) indicate also that measures of personality 
and interest would be of value in selecting and counseling. The 
Strong Vocational Interest Blank for Men is being used as a 
predictor item lor success in dentistry at the University of 
Maryland. However, no published reports are y« available 
to reveal relationship* between scores on tho interest scale and 
criteria of success in this school 

, Test Used 

During 1942-43, three teats were administered to students 
m the College of Dentistry at Northwestern Umveraity 1 These 
tests were 

(1) Preference Record—Form AS, by G, Frederic Kuder 
(3,9,10,17). 
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( 2 ) California Test of Personalty—Adult Senes, devised 
bj Ernest W Tiegs, Welles W Dark, and Leins P, Thorpe 

(16). ,, * * 

(3) MacQuame Test for Mechanical Ability, by T W. 

(rfacQaarrie (II) 

flieii iacQuarrie Test for Mechanical Ability and the California 
TestofPer.ro/udity were administered to 158 freshmen, The 
UecQuame, the California Test of Personality, and the Kuder 
Preference Record were administered to 66 seniors 
Criteria 

Previous studies (7, 8) at the College of Dentistry, North¬ 
men! University, indicated that the MacQuarrie was a usable 
tut for predicting freshman year average grade, Following a 
(Digestion in those studies, an attempt was made in the present 
study to refine criteria for test evaluation by separating theo¬ 
retical and technique grades from practicum grades This was 
done by obtaining cumulative points earned in technique and 
theory and cumulative points earned on product or work done. 
The scores were arranged m ascending order and divided into 
deales It was then possible to score anywhere from 1 to 10 in 
both hmds of grades. 

Results 

Table 1 presants the correlations obtained for the 158 fresh- 

* 

Table i 

Cwnbcbv of Test end Criterion Scorer for l$t Ftefkmen 


Theory ud Technique Pnctjeum 


UkOkkuc TonJ Score . . 
SUf-Adjutmiat Ton' ,, 


men between theory and technique and practicum criterion 
scores and scores on the two tests The California Test of Per¬ 
sonality gives Self-Adjustment, Social-Adjustment, and Total 
Adjustment scores These wore correlated separately. It 


should 1 be pointed out chat the California Personality Test (16) 
has not been validated against objective outside criteria Hie 
1BQ items were evaluated in the following manner 

(1) Judgments of teachers, principals, test experts, person¬ 
nel directors, and employers as to whether or not each item was 
an indicator of adjustment and employability 

(2) The reactions of employed adults as to whether or not 
they judged each item to be an essential characteristic of a suc¬ 
cessful employee. 

(3) The extent to which the results of the test agreed with 
the known characteristics of particular adults. 

(4) The extent to which each item was consistent with the 
score on the test as a whole (bi-serial r) 

Due to the questionable reliabilities of the six components 
in each of the categories Self-Adjustment and Social-Adjust¬ 
ment, it was decided to use only the over-all scores for pur¬ 
poses of group comparison. The split-halva reliabilities of 
there totals are; 

Sec. 1 Self-Adjustment..BB8 

Sec 2. Soaal-Adiiutment..893 

Total Adjustment. 918 

Only two of the correlations approach statistical signifi¬ 
cance For the 158 freshmen there is no relationship between 
total score on the MacQuarru and standings in the criteria 
There u positive but low correlation between Practicum cri¬ 
terion scores and Social-Adjustment and Total Adjustment 
scores on the California Test of Personality, 

Table 2 presents the correlations between criterion scores 
and teat scores for 66 seniors. Only the three of nine corajio- 
n en ta of the Kuder Preference Record on which aeniora averaged 
definitely above the norma for this scale were correlated with 
the criterion scores. 

For both freshmen and seniors (Tables 1 and 2) the Mac- 
Quame total scores have little or no relationship to cither Theo¬ 
retical and Technique or Practicum criterion scores. It is possi¬ 
ble that the first three sub-tests of the MacQuarrie measure 
motor skills and the list four sub-tests measure mechanical 
ability Using total scores might) therefore, obscure rdation- 
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TABLE 1 

Conditio*! of Ten end Criterion Scam far 66 Sudan 


Theory end Techfdqea Piacoran 



ships of cub-testa to criterion scores Total scores were com¬ 
puted separately for the Tracing, Dotting, and Tapping testa 
and for the Copying, Location, Blocks, and Pursuit tests 
These sub-total scores were then correlated with the criteria 
These correlations arc presented in Tables 3 and 4. 

TABLE 3 

CoirtlaltOBj oI MecQeerth Svb-Taul Scow mA Crilenn lor tSS Frakmen 


Theory am] Ted wagns Pnwfcum 

Tndof, Tippifl*, Dottini ... “ }J 

Copyim, Lacadna, Bloc fa, Puisult ,, .16 -U3 

It can be aeen that consistently, for both freshmen and 
remora, Tracing, Tapping, and Dotting scores correlate nega¬ 
tively with Theoty and Technique and positively with Practi- 
aim scores, and Copying, Location, Blocks, and Pursuit corre¬ 
late positively with Theory and Technique and negatively with 
Practicum scorn. Five of these eight correlations approach 
statistical significance. If the first three sub-teste measure 
motor dexterity and the last four sub-testa measure perceptual 
or visualizing abilities, Theory and Technique scores could be 
expected to correlate with the last Tour sub-tests and Practicum 
scores could be expected to correlate with the first three qyb- 


TABLE 4 

CemUitatu d Jfrffffurtf Bearer wiik CnlerU lor 66 Semen 
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testa. This expectation receives support from the positive and 
negative sign directions or the obtained correlations, though 
such breakdown does not provide correlations of sufficient mag¬ 
nitude for use in individual prediction. All correlation! (Table 
2) between scores for seniors >n Self-Adjustment, Social-Adjust¬ 
ment, and Total Adjustment scores on the California Test of 
Personality and Theory and Technique and Practicum criterion 
scores are positive but low. The correlations between Practi¬ 
cum entenon scores and Self-Adjustment, Social-Adjustment, 
and Total Adjustment scores approach statistical significance 
The correlations between Self-Adjustment, Social-Adjustment, 
snd Total Adjustment scores and Theory and Technique ^ap¬ 
proach statistical significance. These findings indicate rather 
TABLE S 

Avemtt PtruniUi Stendinti d 66 Sedan {% Compnenii o) the 
Kndee Preference Beard 


Caraponanti Avcnp Paceatib ObuJaad 



consistent relationships between what tc measured by the test 
and standings in the criteria. However, these correlations 
would nor be useful for individual prediction*. . 

Table 5 presents the average percentile standing of the 66 
seniors jn the components of the Kuder Preference Record. 

The average senior scores above average in Mechanical (91 
percentile), Scientific (93 percentile), and Social Service (67 
percentile) on the Kuder Preference Record, Mechanical inter¬ 
est scores do not correlate with either Theory and Technique 
or Practicum entenon scorer (Table 2). Scientific interest 
scores' correlate positively end significantly with Theory and 
Technique entenon score* but not with Practicum, and Social 
Service interest scores correlate positively and significantly with 
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Practicum criterion scores but not with Theory and Technique. 
These findings indicate that mteteet patterns are related to the 
marks earned in Dental School, but the relationships appear to 
bo more specific than the relationships between personality 
measures and marks earned 

The failure of the scores in the three components of the 
JCitder Preference Record to correlate more highly with cn- 
tenon scores could have been due to the narrow Bpread of the 
scores (see average percentiles, Table 5), particularly in Me¬ 
chanical and Scientific interest scores, It is well known that 
the coefficient of correlation is affected by the variability of 
.scores in the group tested To teat this idea adequately it will 
be necessary to administer the Presence Record to a group of 
freshmen and a group of seniors in a follow-up study 

'Hie deviation of an individual’s interest scores from pattern 
(TaGle 5) may also be an index of the extent to which the scale 
places those freshmen and seniors standing low in Mechanical, 
Scientific, and Social Service interest scores low in criterion 
scores and places those scoring high in these same components 
high,in criterion score* 

To determine whether or not there were statistically signifi¬ 
cant differences between tho mean scores of freshmen and 
remora on the AfacQuarne and the California Test of Pcrson- 
ality, critical ratios were obtained There critical ratio* Indi¬ 
cate that there is no significant group difference between fresh¬ 
men and seniors on the MtxQuarric. There is a significant 
difference in favor of the seniors in Self-Adjustment scores on 
the California Test of Personality. There are 94/100 chance* 
that a difference in favor of the senior* in Social-Adjustment 
■cores is real. There are WOO chances that a difference m 
favor of the seniors In Total Adjustment scores is real 

Summary 

(1) Senior* m dentistry score above average in Mechanical 
(91 percentile), Scientific (93 percentile), and Social Service 
(6? percentile) interest scores on the Ruder Preference Record, 
Mechanical interest scorea do not correlate with either Theory 
and Technique or Practicum criterion score*. Scientific uiter- 
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est score* correlate positively with Theory and Technique cri¬ 
terion scores and Social Service interest scores correlate posi¬ 
tively with practicum criterion scores. 

(2) Statistically significant difference* in favor of the den- 
tistiy seniors over dentistry freshmen in Self-Adjustment, 
Social-Adjustment, and Total Adjustment scores on the Calf, 
forma Test of Personality were obtained Whether or not these 
differences are due to selection, age, training, or all three n not 
known 

(3) Tho California Test of Personality gave statistically 
significant positive correlations between Practicum enterion 
score* and Social-Adjustment and Total Adjustment scores for 
freshmen and between Practicum criterion scores and Self- 
Adjustment, Social-Adjustment, and Total Adjustment scores 
for senior* Statistically significant positive correlations be¬ 
tween Theory and Technique criterion scotcb and Self-Adjust¬ 
ment, Social-Adjustment and Total Adjustment scores were 
obtained for seniors, 

(4) The results of this investigation indicate that correlat¬ 
ing MacQuome total scores with entena may obscure relation¬ 
ships of sub-test scores to criteria 

Personality and interest scale scores show some relationship 
in this study to criteria of success in Dental School, but the cor¬ 
relations are not of sufficient magnitude to be useful in indi¬ 
vidual prediction when selecting applicants for admission to 
the College of Dentistry. 
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THE WOfcD-DEXTERITV TEST, A BETTER 
MEASURE OF COLLEGE 
APTITUDE 

. SHAILER PETERSON 

The Uwveroiy Quo|o 

TBa purpose of thia arocle is to describe a measuring inetru- 
ment which, a* an aptitude test, compares favorably with ex¬ 
aminations commonly given during Freshman Week. This test 
has been tried at junior-high-*chool, remor-high-school and col¬ 
lege level and ha* proved itself a valuable predictor of school 
and course grade* 

Tho origin of this examination date* hom work in remedial 
reading carried on at the University of Oregon in 1933, in coop¬ 
eration With the lata Dr. DeBiuk. Experimentation indicated 
that there w*e a distinct improvement in working vocabulary 
as soon as the student could see the tarry-over in word meaning 
from one word to another. While thia seemed to ho particu¬ 
larly marked in the field of tho natural science*, other work ha* 
indicated that this ability aids student* in other area*. The 
firat mtmimenta were not teat* hut instead were primarily 
teaching devices and intended for students who required reme¬ 
dial assistance to unprove their school marks. New instruc¬ 
tional devices were prepared at Lebanon High School and at 
the University of Oregon High School At this time, the author 
described these fint efforts in The English Journal.' Late*_at 
the University of Minnesota, with die encouragement of Of. 
Alvin C Eunch, Dr. Guy Bond, and Dr. Palmer 0 Johnson, 
more experimental work was conducted on this same project. 
Still more recently at South Dakota State College there wv 
further opportunity for the author to obiervo the value of the 
WortLDexterity Test that had been developed. 

* PtuiiML SbaBcr A Tndikw Vonbehri** TMKudptJW- 

ui (CpUr*« jwWo"), xxv etmj.lw*. 
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THE WOED-pBXTBRlTV TEST 


Ths main objectives of the teat in its present form are to 
jgftnnlna to what extent the student knows the meaning of 
nrtun suffixes and prefixes in common use and also to deter¬ 
mine if bo can transplant the meaning of a suffix or prefix found 
w one word, whose meaning ib known, to another word in which 
triune suffix or prefix is found From an examination of the 
Htt items illustrated in this article, the reader can understand 
tfct while pure memory of words, suffixes, and prefixes will 
loiit the student in securing a high score, still when the range 
gfword difficulty Is at the proper level, the test becomes easen- 
iiifly a problem for the student to demonstrate hu ‘'dexterity” 
it manipulating word parts and word meanings. Mechanically, 
tho also becomes an “analogy” type tut or 6no which testa for 
tooEguration and pattern, 

From an examination of the following directions and sample 
rett items, the reader will be able to understand the construc¬ 
ts of the entirt test. It is a power test rather than a speed 
tot and the fifty-item test described here could be administered 
tp samor-high-school or college students in 40 minutes. The 
jotdiused m the examination were chosen after consideration 
of -their Thorndike word count These words, in addition to 
giving a fairly extended range oF difficulty, all employed suffixes 
md prefixes which in turn were to be found in at least four or 
Sre very common words as revealed by their Thorndike word 
count, v 

Dtrictionj 

Tn tfui tear, there arc many scientific wurdi that you already 
know Of the others, you will find that in many cases you will be 
lUa to dlicover their meaning as you proceed, 

la the example, the word part SUB is considered. There are 
Btny words in our English language containing SUB Ono of there 
MtJf, SUBMARINE, has been printed on the Une alongside. There 
|i ■ spaa for two other words containi ng th is same ward part, SUB, 
uid ui this example, the two words, SUBWAY and SUBNORMAL 
have been written In. 



Each of the three words, SUBMARINE, SUBWAY, and SUB¬ 
NORMAL, contain the same common part SUB The fint word 


refers to an imdre-water boat, the second word refers to an «nd#r- 
gronnd railway; while the lut ward refers to things or conditions that 
are below or under normal All three of these words not only hive 
the same word part, SUB, but they all have the arme meaning, 
"under 

On another line bcaide each of the lecrions, there Is a group of 
denaidane, one of which n correct. You are to select the correct 
definition and place us letter in die apace in the margin, Aa you will 
see in the following part of the example, definition "B” best describes 
the word part, SUB 


rasa . 1 


lltto, aSMUtati Is) InuUi, Mn>. 

l®> do, 


While there is e space in each section for you to write only two 
additional words, you may be able to think of more than two, In 
aome care* you may have difficulty in thinking of more than ono ad- 
ii°i 006 more words that you tan think of, the easier it 
Will be ta deoda what the true meaning is, far ai in the cue of the 
words submarine, subway, and subnormal, each woid is alike in two 
respects There words have the similar word part SUB and also 
have the similar meaning, “under" 

You will And that this Is true of other words also. Whether you 
can think of five new words or only one or cwo, try to deade on the 
best definition for the word pan under cons Id era don. 

DONT WASTE TIME. DO FIRST THOSE ITEMS 
WHICH ARE EASIEST FOR YOU. 


3. MIS 

4 CO 

S. 1ST 


6. LOGY 


Ten Items 
misunderstand 


(A) wrong*, (B) inull, tiny, petty, (C) ex¬ 
cuse, reason; (D) denotes fenunma gender; 
(E) poor, inferior 

cooperate 1 _ _ 


(A) with, together; (B) two, (G) vntl 
stopping, (D) aid, kelp; (E) easy, ready 


(A) science, logical study, investigation, (B) 
schooling, training, preparation; (C) forming 
agent nouns; (D) helping, assisting, aiding; 
(E) including, possessing, Raving. 

mineralogy ___ _ 

(A) weather, climate, Temperature. (B) 
lng| (C) prediction, estimation, (D) disci 
theory, doctrine; (E) bred, Weary, slow 


310 EDUCATIONAL AND PSYCHOLOGICAL MEABU1EMENT 
_10 GRAPH biography _ 

_11 ANTI 

_32, 8CRTB 

_34. POLY 

_39 GEN 

_44. PATH 

_47. DECI 

_48. DUCT 

__49. ANTE 

In these items, the student is asked to write down a group 
nf other words each of which contains some of the same prefixes 
and suffixes. This U to a*»lit him in assigning wort^meaning 
to the word parts In the unknown word, The test is scored, 
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however, on the basis of correct responses to the word's mean¬ 
ing and ordinarily no attention is paid to the particular “assist u 
words that he writes down. If this examination were adminis¬ 
tered in a situation where attention was to be given to remedial 
work, then the character of these "assist” words would be 
Important. 

Table 1 shows die product-moment correlation coefficients 
between different vanates in a college group of nearly three 
TABU 1 

CcmUifoN Ceifieienir between Mem il 7eri*l*t to CotUi* Grant 



hundred atudents From this table, it is evident that the Word 
Daxtenty Test predicts grade-point average, English grades, 
mathematics grades, or chemistry grades better than most of 
the other examinations with which it wai compared- Grades 
themselves m some subjects were better predictoie of total 
grade-point average, but this can be explained by die fact that 
the individual course grades were themselves a part of die 
grand avenge. 

At die high-school level, the product-moment correlation 
between the Word Dexterity Examination and LQ. was +^8 


(A) living dungs, life, (B) Important, 
guuhed; (C) soil, ground, rocks; (D) 
trig, writing; (E) a study, a science 


distm- 

draW- 


(A) treatment, medical aid, (B) old, histori¬ 
cal, not used; (C) in front of, before, preccd- 
ing| (D) opposite, agajnst, Instead ol) (E) a 
study, a science, 
transcribe 

(A) radio, phonograph; (B) circular, round, 
(C) write; (D) duplicate, (E) vole*, vocal 

polytechnic _ _. 

(A) arithmetic, numbers; (B) many, much, 
often; (C) few, not many, (D) figures, dia¬ 
grams; (E) college, school, study. 

genealogy ^_ _ _ 

(A) suenoe. ttudy; (B) birth, boni, descent; 
(C) related, corresponding; (D) electrical; 
(E) Bible, Biblical 
patholosy 

(A) scientific, (B) anger; (C) suffering; dis- 
earo, ID) Judgment, estimation, (E) germs, 
bacteria. 

decimal . 

(A) anthmedc proeaM, multiplication, etc.; 

(B) 1 half; (C) ten, ten dines, (D) almost, 
nearly, quite, (E) accurate, scientific, sound. 

ductile 

!Lr- ■ 

attracted, attractive 

■ntndatn 


save, preserve; (B) stretch, lengthoi, 
i. Batten, uuceu; (D) able to be 
, (E) lead, direct, guide. 


(A) outside, beyond; (B) behind, beside, (C) 
in front of, preceding, before; (D) old, histori¬ 
cal, not used; (E) not, never, none. 
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find with menial age, it wm + 70. The point bisenal correla¬ 
tion* with years of sderce wsb + .20 and with yean of foreign 
language was + 39^The bisenal correlation with science grades 
was + 52. 

While the scores for the 7th- and 8th-grade students are 
considerably lower than thoio for any of the other groups, it is 
interesting to observe that there is considerable overlapping of 
the scores for the upper grades Figure 1 illustrates the per¬ 
centile rank curves for the seven grade levels. 



Pitcmlilt XjaS 
Flora* 1 

PacflM&i Rink Con* lax ?nmm Weri JJaioray Tat 


The estimated reliability of the forty-minute Wonl-Dex- 
terity Test was + 90 to +. 93 depending upon the group* These 
computations were made by the Hoyt method* and by Formula 
20 of the Kuder-Richanlson method,* each of which gives 
underestimates of the true reliability, In order to evaluate the 
function of each of the items in the examination, the item-test 


fa In ihcM U 

*Hoyr, Cyril *Tk« RdmMiiy Eiranjttd by Amlyip J Varanra." Piytko- 

W,r ^lchii*Jn!'M S Knden G F ‘Tlu GUwlarinn of Tat RduUity 
Codfinont* Bittdwi *»RitlMil Enurraltara." Mnwf of ICkmHomI 
PtycJtfaty. XXX (1919), <83-681 


TIM WORD-DEXTERITY TEST 


correlation For each of the items was computed both on the 
examination administered at the high-school level and on the 
one administered at the college level Table 2 provides a fre- 


TABLE 2 

DLlrihUvn of Iltm-TM CowffltfeiU Bawd ujoit Taj, a+d Botttr*. QvttUrt 


Item-Tot Can datum 


+ 8 to 9 
7 to J1 
Am 7 

J to 6 

4 to S 
3 m 4 
2 to J 
I to 2 
U to 1 

Medan Value of r 


Frtqococy of Ilum 


Hlch-Schoal Group 


Collni Gmap 


quency distribution of the fifty items according to the correla¬ 
tions that they were found to have with the total test, a mea¬ 
sure of item validity when the total teat is the criterion measure. 
The high item-test correlations for all Items m the examination 
must be interpreted to mean that there is a commonality of 
purpose and function for all items in the examination and that 
whatever is being measured by the enure examination is being 
contributed to materially by each item in the examination. 

Sample copies of the author's Word-Dsxtcrity Tut are 
available directly from him and permission to duplicate this 
examination can be secured. 
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A STUDY OF THE KUDER PREFERENCE RECORD 

DANIEL J BOLANOWGH ami CHARLES H. GOODMAN 
Radio OBfponnou d Amina 

At the beginning of die war, one of the major problems that 
faced industry was the need for highly trained technical per¬ 
sonnel, The Radio Corporation of America, like many other 
companies faced with a shortage of engineers, planned to meet 
this need with a abort intensive training program for young 
women The program sponsored by RCA was a ten-month 
coqtse m the theory and practice of electronic engineering* and 
was given at Purdue University 

Because of the importance of obtaining a maximum number 
of technical personnel who would he successful in the course, 
psychological tests were used as part of the selection process. 
The findings which were obtained as a result of the use of theae 
psychological testa will be published at a later date. 

Three hundred applicants from various colleges of thirty-suc 
states and from RCA’s atx manufacturing plants applied for the 
course. Of this number eighty-six were finally selected The 
girls selected were called RCA Cadettes Their ages ranged 
from eighteen td twenty-nine years, with a median age of nine¬ 
teen years. Frvo Cadettes had hod no previous college experi¬ 
ence, sixty-five of the group had either one or two yean of col¬ 
lege work when they entered the course, and one Cadctte had 
had graduate work. 

pie positions which the Cadettes were expected to fill were 
varied and included such activities as writing instruction man¬ 
uals, design, drafting, statistical analysis, laboratory assembly, 
and equipment testing Because of die variety of Jobs open to 
the Cadettes It was felt that it wquld be helpful m placing them 
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if their interest! could be determined. One month after the 
Cadottea had been at Purdue Univenity the Kuder Pnfennct 
Record was administered to them The reasons for using the 
Kuder were' (1) it offered interest areas which appeared 
related to the jobs that were to be filled; (2) it offered possi¬ 
bilities of determining the preferences of the Cadettes for plao- 
roent, (3) it offered on opportunity to examine the Kuder as a 
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possible selection device for future training programs, and (4) 
it was easy to score. 

The results reported in this paper are hosed upon the scores 
obtained from the sixty-six Cadettes for whom there were com¬ 
plete data Upon completion of the scoring of the Preference 
Ricordj an mdlviduol profile was constructed for each Cadette, 
In order to obtain a graphic profile that would be represents- 
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^ of the Cadettes as a group the median acoree were calcu¬ 
lated for the Cadettes on each category of the Preference 
Record. 

Figure 1 shows the profile constructed by plotting these 
median scores after converting them into the percentile equiva¬ 
lents given by Kuder in his norms for college freshmen 1 It is 
Interesting to note from Figure 1 that the Cadettca as a group 
itand at the 84th percentile in the Mechanical preference area 
and at the 92nd percentile in the Scientific preference area 
There is, of course, the possibility that the Cadettea' responses 
may have been influenced by their desire to appear highly mter- 
mted in scientific and mechanical pursuits because of the nature 
of the course they were taking. It should be remembered) 
however, that the Kuder Preference Record played no part in 
die selection process of the Cadettea. The Cadettes appear to 
be similar to the norm population in Computational, Artistic, 
ind Social Service preferences. They are, however, consider¬ 
ably below the median of the norm population in Persuasive, 
Clerical, and Musical preferences. 

The graphs in Figure 2 show the percentage distributions 
of the Cadettea 1 scores on each of the preference categories 
The norm graph shown was constructed in order to facilitate a 
visual comparison of the percentage distributions found for the 
Cadettes on each of the preference categories. The base lines 
nf each graph have been divided at interval points correspond¬ 
ing to the 30th, 25th, 50th, 75th, and 90th percentiles for the 
norm group Each graph can then be compared to the norm 
graph. 

Figure 2 appears to support the evidence of Figure 1 of the 
selection of a group particularly interested in the Scientific and 
Mechanical categories, a finding similar to that obtained by 
Goodman* for engineering students. Thera are, however, a few 
Cadettes below the norm medians on these keys The distri¬ 
bution of the percentages on the Clerical graph is the reverse 
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of those on the Mechanical and Scientific categories Forty- 
four per cent of the Cadettes fall below the 10th percentile and 
only four per cent are above the 75th percentile. The majority 
of the Cadettes Fall below the 25 th percentile in Persuasive 
preference, although there are five per cent above the 90th per¬ 
centile who might fit into jobs where considerable contact with 
oihera is necessary, auch as quality control inspectors. The 
Cadettes tend to fall in the middle ranges in Computational 
Preferences 

These graphs were useful in helping to determine finafplace- 
raent of the Cadettes. |Th® Cadettes in the extreme ten per 
cent groups of each Preference category were given particular 
attention for possible placement m jobs related to their prefer¬ 
ences. For example, some of tho jobs to be filled involved 
mechanical work, scientific work, clerical work and literary 
work The histograms in Figure 2 were also used m counsel mg 
the Cadettes and aiding them ui the interpretation of their own 
individual profiles 

To evaluate the possibilities of the Preference Record as a 
selection cool for similar programs, correlations of the Cadettes 1 
Preference scores with their final grade avenges for the entire 
course were obtained. It was found that first- and second- 
acmes ror grade averages of the Cadettea correlated .81, which 
would indicate fairly close agreement between the grades of the 
first and second semesters. Aa a result total grado averages 
were taken as the criterion. Table 1 gives the frequency du- 
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tribution of grade averages, and shows a range from 3 0 to 5.9. 
The grades for each subject were given in whole numbers rang¬ 
ing from 2 to 6. Grade averages were obtained by multiplying 
each subject grade by the number of credit hours and dividing 
the total for all subjects by the totil number of hour* Tor die 
course 

Table 2 shows the correlations of grade averages with the 
venous Preference scores. Since none of the correlations attain 
statistical significance* (even at the five per cent level) the 
Preference scores cannot be considered as predictive of success 
m the Cadette training course. 

For further clarification of the relationship between Prefer 
ence scores and success in the training course, an analysis was 

TABLE 2 
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made of the scores of three sub-groups—(1) the most success¬ 
ful students, (2) the least successful students, And (3) those 
who terminated thelrjraming before the course was completed 
Figure 3 graphically shows the median scores converted into 
their percentile equivalents on the Kuder norms of 16 Cadettqs 
with grade averages of 4,9 or above, 13 Cadettes with grade 
averages of 3Jf and below, end the 8 Cadettes who dropped out 
of the course for reasons other than those attributed to illness. 
For groups as small as these, the proximity of the curves for the 
most successful and least successful Cadettes who completed 
the couree appears pronounced. Figure 3 aeemi to bear out the 
low correlations of Table 2 The largest differences between 
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the most successful and the least successful Cadettea appear to 
he on the Computational, Artiste, Musical, and Social Service 
categories 

The profile of the terminated Cadettea appean to deviate 
from the patterns of the groups who completed the course. The 
terminated Cadettea have lower median percentile scores in 



Social Service categories. 

In order to determine the significance of the differences 
shown in Figure 3, t ratios were computed using Student's tech¬ 
nique for small sample distributions Table 3 shows the median 
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raw scores Tor the three sub-groups on each of the Kuder prefer- 
enco scales The t ratios obtained for each group, when com. 
pared with the other two groups and the probability values for 
these t ratios, are shown in Table 4 

Accepting Fishers criterion of a 5% probability as indica¬ 
tive of a significant difference, n will be found that there are 
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only five such P values In Table 4 The five instances of sig¬ 
nificant differences are 

Computational-—Most successful Cadettea icor signlfir 
cantly higher than the least successful 
Cadettea. 

Com putational—Most successful Cadettea score signifi¬ 
cantly higher than the terminated Ca- 
dettes. 

Scientific— Most succmfui Cadctta score signifi¬ 

cantly higher than terminated Csdetta. 

Persuasive— 'Terminated Cadettea score significantly 
higher than moat successful Cadettea, 

Persuasive— Terminated Cadettea score significantly 

higher than least successful Cadette*. 
The results indicate that the most successful Cadettea are dif¬ 
ferentiated (ram the least successful Cadettea on the Computa¬ 
tional Jctsy only The most successful Cadettea are differenti¬ 
ated from the terminated Cadettea on the Computational and 
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Scientific keys The terminated group is differentiated from 
the most successful and least successful Cadettea by higher 
scorn on the Persuasive scale. On the whole the group of 
terminated Cadettea appears to approximate more nearly the 
pattern of preferences of Kudcr*i college freshmen 

The above differences provide the only evidence that may 
indicate any possible utility of the Kuder Preference Record aa 
a selection device for future applicants in a training course such 
as the one described, 

Summary 

Thi« study has presented data based upon the Kuder Prefer- 
•net Record scores of 66 Cadettea enrolled in a training pro¬ 
gram for electrical engineering aides, Investigation was made 
of the possibilities of the Kuder Preference Record as a selection 
device and us usefulness as a counseling and placement tod] 

The findings show that the Cadettea, as a group, differed' from 
the college freshmen norm group, having particularly strong 
preferences in Mechanical and Scientific purauita There was 
a pronounced lack of preferences in Persuasive, Clerical, and 
Musical endeavors. Score distributions showed, however, wide 
differences among the Cadettea on each scale. There were, 
however, a few Cadettea who showed a lack of preferences m 
Scientific and Mechanical fields while a few ranked very high' 
in Clerical, Computational, Literary, and Persuasive prefer¬ 
ences. These distributions, together with the individual pro¬ 
files, provided some indications Tor counseling and placing the 
Csdettet. 

The acale* of the Preference Record showed low correlations > 
with final grade averages for the course* The highest condi¬ 
tion was .18 for the Computational key with fjnsl grade ever*' 
age. ; 

The profiles of the most successful Cadettea and the leapt 
successful were similar. The group profile of the terminated 
Cadattes approached that of the average college freshms*^ 1 ' 
These terminated Cadettw made significantly lower scojeq on - 
the Scientific and Computations! preferences than the most 
successful Cadette*. The terminated Cadettea made sigplfh 1 
cantly higher scores than die most successful and least sucttt»? ^ 
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jjj Cadettea on the Persuasive scale. The moat successful 
Odettes made significantly higher scores than the least suc- 
gtffu) Csdettes on the Computational key 

The following conclusions appear to be justified 1 

( 1 ) On the bans of the correlations with total grade ayer- 
qd, the Kuder Prefe roue* Record does not appear to be a 
MDffliring selection device for predicting course achievement of 
{disk engineering Cadettes It does, however, show some 
munise as a device for eliminating those who would be likely 
todiop out before completion of the course 

(2) The Kuder scores afford Some indications that can be 
^pful id counseling and placement, especially in a situation 
Hhtrethere is a variety of joh openings 
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Our national survey showed that service rating has proved 
of real value in raising civil service standards It is desired 
generally by competent employees since recognition may be 
given to Faithfulness end ment as factors in promotional exam¬ 
inations as well as in eligibility for salary increases, transfers, 
leaves of absence, and other civil service privileges. 

From the appointing officer's viewpoint, service rating pro¬ 
vides an impartial yardstick by which he can measure employee 
performance, and also provides a definite incentive by which 
the majority of employees seek improvement, to that extent 
bettering the performance of hu department and at the same 
□me simplifying his personnel problem. From this definite 
national consensus, we felt assured in following through with 
development and inauguration of our own system 

We also found that regardless of the form of the rating 
device there are, In the opinion of administrators generally, 
certain requisites to the success of any employee rating plan* 

Ftfiv The supervisors or executives who are asked to me 
their subordinates must be sold on the idea- This result may 
be brought about by a series of conferences with such super¬ 
visors conducted by officials of the personnel department. 

Second The employees themselves should be sold on the 
idea. When fnerit racings are properly sold?investigator* re¬ 
port that a large percentage of employees like to be rated and 
will give complete co-operation to any rating plan that they 
consider to be fair. 

Third' The supervisor* must be’trained to rate thaw under 
them. One of the principal weaknesses of rating systems ap¬ 
pear* to be the lack qf umfotmity 6f ratings between different 
departments or bureaus In one department, for example, a 
supervisor may set very low standards of performance and rate 
B5% of his employees “excellent" and the remaining 1?% "very 
good " In another department the supervisor may set a very 
much higher standard of performance and rat* 85% of hu Em¬ 
ployed good and distribute the other 15% anywhere along the 
Kale from excellent to unsarisfactoiy. In attempting to *ulVo 
the problem of maintaining te*sonabl* uniformity of service 
rating standard*'some jurisdictions—notably-; the United State* 
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The “San Francisco System" of service rating was put into 
effect by the Civil Service Commission of San Francisco oo 
July 1, 1944, for probationary employees, and shortly will be 
applied to nearly 15,000 employees of the County and City of 
San Francisco. Before the system was developed and adopted, 
a comprehensive survey was made of various system* in use. 
This paper is concerned with a presentation of the findings of 
our survey and a description of the approach finally taken in 
developing the “San Francisco System." 

Our research took many months, covering most of the major 
federal, state, and municipal personnel jurisdictions throughout 
the United States and Canada We received the finest sort of 
co-operation almost uniformly, and in our analysis of the differ¬ 
ent systems in use, criticism is not intended We analyzed each 
in relation to the specific problems facing us in developing a 
rating system that would be of the most value for our own local 
jurisdiction 

The problem of rating employee performance is aciil m the 
experimental stage and, up to comparatively recent times, rat¬ 
ings were considorod more or less unreliable Personnel agen¬ 
cies have lately devoted a very large amount of research to the 
problem, while leading personnel writers and publications have 
exhaustively gone into the field 

From San Francisco's viewpoint, tha purpose of inaugurat¬ 
ing a service rating syaten was to develop a means of attracting 
more capable worker* into public service as a career, raising the 
caliber of city personnel, m lino with the merit system trend 
to further eliminate the “spoils system" from public service. 
Such a service rating system was authorised by our city charter. 
m 
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Housing Authority, the Tennessee Valley Authority, and the 
New York State service—have adopted a plan based on the 
idea that the categories of “excellent" and "unsatirfactoiy" 
should bo reserved for those employees who distinguish them¬ 
selves from their fellow workei* unmistakably, to either the 
advantage or the disadvantage of the service, and when such 
ratings are given, tha supervisor is required to cite on the back 
of the rating form, or on a special form, concrete evidence of 
such service 

Fourtkt The rater should take the ratee into his confidence, 
show him his ratings, and discuss them with him When the 
employee knows what traits or qualities ere judged to be un¬ 
satisfactory, his normal resetipn would be to seek self-improve¬ 
ment Especially will this be the com when ratings are used 
as a factor m promotional examinations, and also in determin¬ 
ing the order of layoff and reemployment, and eligibility for 
salary increases, transfers, leaves of abasnCe, and other privi¬ 
leges. How the employee does his work must be determined 
in most "Cases by the employee's immediate supervisors One 
of the problems therefore is to secure reliable ratings from vari¬ 
ous supervisory officers. In this respect there are two schools 
of drought. One main tain* char the actual assignment of a 
rating should be made by the supervisora who are direfctly in 
charge of the employees The other and more recent school 
coniine* the supervisor to the reporting or significant behavior* 
or activities, relying for the actual valuation upon some me¬ 
chanical scoring system administered by the central personnel 
Staff 

1 In deviling a rating system the two principal technical 
problem* have to do with the selection of factor* to ha rated 
and the comparative, evaluation of auch factors. Authorises 
generally are agreed that the objectivity and reliability of rat¬ 
ings Increase as the factors involved become more specific. The 
chief fault to be found with rating plans in the past Is that atich 
plana provided for over-all raring* by lh° wpmrfpot- Fre^enc 
practice is Opposed to reliance uppn iiiqb highly‘’subjective refl- 
mptes. It prefen, overwhelmingly, rarihga which, are objec¬ 
tive. The maximum degree p( objectivity Is approached if the 
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attention of the supervisor is focused on specific job behaviors 
or activities, In addition, each item to be reported or evalu¬ 
ated should bo dearly described in simple, unambiguous terms 
and as concretely as possible 

The point should be stressed, however, that no matter how 
carefully factors have been selected and no matter how care¬ 
fully supervisors observe and report the behaviors of their 
employees, the whole system will break down unless the actusl 
rating; based on these u sound Almost without exception, in 
the opinion of Mosher and Kingsley, the devices that have been 
developed to evaluate employee services Have failed to provide 
an adequate measuring instrument. 

One method of rating the efficiency of employees, especially 
in industry, is through the use of production records. Such 
poaitioni u stenographer, typist, file clerk, machine operator, 
Dr copyist lend themselves to unit measurement. In some of 
the larger federal establishments, such as the Fam Credit Asso¬ 
ciation and the Federal Reserve Bank of New York, the num¬ 
ber of pages typed, the number of errors, and the general 
appearance of the work are taken into consideration by super¬ 
visors when typiats art rated. 

Another method is through the use of periodic teats. This 
method hta been extensively employed in tome of the federal 
bureaus for rating die efficiency of their employees. In the 
Department of Justice, for example, the service ratings of 
stenographers and typists depend in pan upon their standing 
in periodic speed teats, while performance on a periodic sorting 
test is one basis far determining the rating of postal clerks 
This method-pf rating, however, is adapted only to routine and 
repetitive jobs. 

The method In moat general use is that of rating schedules, 
A description of some of the schedules in use id other jurisdic- 
Honr follows 

]. Ths Graphic Rating Scale, Thu, according to many 
personnel experts, u the moat popular rating method in use, 
being widely employed in private concerns as well as by a num¬ 
ber of public personnel administrations 1c consists essentially 
of two elements; (1) a lilt of traits or activities arrived at by 
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an analy.i. of factora leading to aucceu or mating b, 
on the job, (2) various dwmptive phnaea or adjtmiya deooi 
ing the several degnea of a particular activity or oat Tvl 
form of the device is aa follows. 
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The descriptive phriaea serve as a guide to the rater who b 
instructed to place a check mark along the line in one of fa 
10 boxes above the phrase or between the phrases at the point 
which, in the racer's opinion, represents the degree of the par¬ 
ticular quality possessed by die ratee. 

This method ts open to the objection that there is no stand¬ 
ard unit of measurement involved and the device is not really 
a scale at all. It is a schedule the various items of which are 
arbitrarily weighted and given a numerical value through the 
use of a scoring stencil The device is scored as a rule by the 
application of a ten-point stencil to the straight line. 

2. The Probst Service Ra(mg System, This system, davtl- 
oped by J B. Probat, the Chief Examiner of the St Paul Civil 
Service Commission, in 1928 was in the opinion of personnel 
administrators the only system which up to that time showed 
any real marks of merit. It has been tnod out in a number of 
civil service jurisdictions including the Cincinnati, Detroit, Los 
Angeles, and California Sure public services. Mr. Prohit dis¬ 
carded the scale idea previously mentioned and sulntixuud for 
it a list of about 100 characteristics or modes of behavior The 
rater checks only those items which are known to describe the 
rtteej the rest he leaves blank. In addition, he is not required 
to measure relative degrees of a quality m the ratee Accord¬ 
ing to Its author the scheme is so designed chat failure on tha 
part of the scoring official to correctly and conscientiously check 
the employee's traits can be ascertained at a glance. It hii 
the added advantage that elaborate instructions to reporting 
officers are unnecessary. 
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The actual evaluation ol .employee service is made by the 
use of a mechanical scoring system administered by the central 
personnel office. This scoring system hni been one of the prin¬ 
cipal bases of criticism of the plan because of the difficulty in 
understanding it. 

3. California Report of Performance Plan. In 1938 the 
California State Personnel Board abandoned the Probst Sys¬ 
tem then in use for one which, in the opinion of the Board, more 
satisfactorily and adequately measured the performance of 
State employees. A separate report sheet for different types 
of work containing a particular combination of work character- 
isdci was developed At the present time 45 different report 
forms each containing a separate Jilt of factors to be rated are 
employed. There are five possible gradations of markings for 
each item, represented by five columns These columns are 
defined in terms of the extent to which an item is characteristic 
ol the worked an employee. The individual id highest author¬ 
ity who is in intimate contact with the work of the employee 
acts as tho “reporting” officer. After the reporting officer has 
prepared the report he must review it and discuss it with the 
employee, who is also given a copy ol the report. The original 
is sent to the personnel agency for scoring. An appeal board 
has likewise been Set up to hear and decide cases in which tbe 
supervisor and ths employes disagree on the markings of a par¬ 
ticular report. 

4 The Lot A-ngtlei City Schools recently adopted a per¬ 
formance report system of the trait-rating type where four 
separate forms gre used. A unique feature of this plan is that 
It was developed by representatives of employee groups and 
eiganlrations. These groups studied and discussed the prob¬ 
lem for about a year before submitting the final plan to the 
personnel board. The forms are so compiled that they may be 
machine scored- These two features, and especially tbe first, 
rpighe well be given favorable consideration if and when a rating 
system for a]l employees is put into effect Employee support 
would thus be assured, without which. It is claimed, no plan can 
ftieceed. 

5. The Sea Francisco public school are at present using* 
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raang form for non-certified personnel, It consists of a three- 
step rating Schedule in which three degrees of each trait corre¬ 
sponding to excellent, good,' and 9°° T ate identified by appropri¬ 
ate descriptive phrases arranged in three columns Such a per¬ 
formance report fs open to the objection that the ratings are 
too broad or, in other words, disc they do not give a sufficiently" 
fine graduation of the relative excellence of the employees, 

6. The Home Owners Loan Corporation utilizes a service 
rating form that consists of three traits which are rated directly 
by the rating officer on the bans of excellent, very good, good, 
fair, or unsatisfactory. This method is open to objections, 
among them these! the traits to be rated are too few to yield 
a comprehensive picture of the employee'! ability, alio, the 
traits are compound ones and require an over-all rating of more 
than one type of activity, 

7 The City of St, Low, Department of Personnel, veiy 
recently adopted a service rating plan according to which em¬ 
ployees are rated on eight specific traits and an over-al] evalu¬ 
ation of work performance. These nine measures ate felt to 
include all factora necessary tp arrive at a comprehensive evalu¬ 
ation of employee performance and to apply to all typo of posi¬ 
tions, Thu plan, like that of the H 0 L G, is open to the enu- 
cism that the rAings are too subjective since the traits are 
rated by two different supervisor* who place a check ms A, 
In one of five columns headed excellent, veiy good, good, fair, 
and poor The subjectivity of the ratings u lessened tu some 
extent, however, by a provision whereby tbe supervisor is re¬ 
quired, when a rating of either excellent or unsatisfactory on 
the over-all evaluation is given, to furnish substantiating evi¬ 
dence of the employee's superior or inferior performance on the 

job, 

8 The Detroit municipal sennet rating report consists 

a three-step rating schedule in which degrees of poor, satisfac¬ 
tory, and shove average of 25 traits are rated objectively with 
the aid of descriptive phrases which identify the various degrees 
of the trait Ten additional traits are rated where the employee 
is in supervisory charge of other employees' work. It appears 
to have considerable merit. The number of traits rated objefr 

* 
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ij^jy ]f sufficiently large to give a comprehensive and reliable 
•faint of the employee's worth. The one criticism that might 
[e leveled at the method of rating is that only three degrees of 
trait are rated in place of the conventional five-step rating 
idiedulc- 

9, The Minnesota State report of employee performance is 
u adaptation of the California State employee performance 
fan, It consists of a large number of traits (about 60 m all) 
winch may be measured objectively in terms of the frequency 
wuh which these traits can be observed The traits are checked 
■ one of five columns Different forma are used to rate differ- 
(■t groups of employees. Complete instructions for use of the 
ptuig form are contained on the reverse side of ,the form The 
[uga number of traits to be rated makes the scale rather cum- 
IgiFHne; it has the merit, however, of giving a very compre- 
tailve picture of employee worth 

10. Saginaw, Michigan, uses an employee service report of 
die graphic rating scale type Eight traita are raced objec- 
ovely with the aid of phrases descriptive of degrees of the trait 


math m the opinion of the rater The form has the merit of 
otrtme simplicity and allows the rater wide latitude in his 
appraisal of the degree of each trait possessed by the racee. 

11 A distinct departure from the conventional rating 
schedule in general use is the "Employee Guidance Sheet," as 
his called, recently developed by the Alabama State Personnel 
Difmrtmsni. The honzontal scale has been discarded in favor 
of a check-list arrangement The five descriptive phrases 
identifying the various degrees of the ten traits to be rated are 
ouched in language intended to help and encourage the em¬ 
ployee rather than to report findings in a coldly impersonal and 
noenmes quite blunt manner. The following sample will illus¬ 
trate the effort that was made to humanize the* report and 
itunulate the employee to greater effort 


Usual Form 

Quantity of work. 

! ) Unuiually high output. 

) High output. 

1 Normal output, 

1 Limited output. 

) Insufficient output, unsatisfactory 
Alabama State Form 

! Quantity of work 
Jliat a friendly suggestion, 

) Exceptionally high output. Keep it up. 

) Better than avenge. Good going. 

) Meeting our requirements, 

) You could do more Try harder. 

) You muld do a lot more. Try much harder 


Other features include a statement in which the reporting officer 
indicates whether the duties of the position have changed since 
the last rating period and also a scale on which the department 
head indicates his opinion of the supervisor’s ability as a rater, 
All in all, this rating form seemed by far the best of any studied 
by us. It combined all the features of an exceptionally well 
developed rating system, including simplicity, ease in scoring 
and rating, and high objectivity and reliability. In addition, 
the language employed in the rating scale was so chosen that 
the employee would feel that the criticism was offered in a help¬ 
ful mood and in consequence should have a strong desire to 
improve his performance if it were below normal. 

The foregoing discussion is devoted to a review on a very 
limited scale of representative rating plans in use None of 
these plans are restricted to the rating of the probationary 
period alone. In inaugurating the San Francisco employee 
rating plan, therefore, it was felt desirable, in order to acquaint 
employees and employers with the idea gradually, to limit it to 
probationers first The San Francisco charter provides that "at 
any time during the probationary period the appointing officer 
may terminate the appointment" Since power of dismissal is 
vested in the appointing officer and since hn decision is final, 
we felt that simplicity should be the determining charactenitic 
of any rating plan for such employees Keeping in mind the 
objectives that should be inherent in every good rating device, 
we attempted to develop a plan which included the best fea- 
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tnret of those reviewed, with particular emphasis on the follow¬ 
ing points* 

1 The development of a uniform report sheet containing 
traits which wQl fit all types of work. 

2. Clear and concise wording of the descriptive phrases 
which measure the degree of each trait 

3 Phrasing of steps so as to compel supervisors to give care¬ 
ful consideration to their markings. 

4, A sound and easily applied scaring formula, 

Hie San Francisco rating plan utilizes a simple check form 
applying uniformly to all similar classes of positions in the van-i 
oua city departments, with the exception of the Poliee and Fire 
Departments, for which separate forma have been prepared. 
Hie form u filled out by supervisors and double checked by 
department heads or appointing officers. 

The traits included in the form which apply to all employees 
ire as follows Promptness, Attendance, Quality of Work, 
Ability to Learn, Co-operation, Dependability, Judgment and 
Initiative Additional traits which apply only to certain jobs 
sit' Volume of Work, Contacts with Public, Physical Fitness, 
Appearance, and Initiative In the case of supervisory posi¬ 
tions, Ability to Tram and Organization of Work are included 
The five phrases descriptive of the degree of each trait are 
arranged in the form of a check list, and excellence on the scale 
u rated sometimes at the beginning and sometimes at the end. 
Such an arrangement increases die probability that the rater 
will pay full attention to the descriptive phrases, 

On the back of the report is space For the certificate of the 
reporting officer and of the employee involved. The appointing 
officer in cases involving probationary employees is required to 
indicate his impression of the employee’s General Fitness to 
bold the position, and to indicate presence or absence of unde- 
nrable characteristics which would make the employee unsuited 
to the particular job. If the probationary employee is rejected, 
the appointing officer must indicate specifically why. Ratings 
ire given probationary employees at end of the second and fifth 
months of service, and give appointing officers a legitimate 
excuse for rejection of an unsatisfactory applicant by refusal 
to certify him for permanent tenure, 
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The probationary penod lb a definite part of the examina¬ 
tion and tests those factors for which no adequate written or 
oral tests have been devised Unfortunately, however, appoint¬ 
ing officers often fail to recognize their responsibility in checking 
the probationer's services and too often the probationary period 
elapses without adequate investigation of the employee's per¬ 
formance on the job. It is hoped that the use of the service 
rating system for probationers will serve to impress upon the 
appointing officer his responsibility in determining whether or 
not the probationary appointee will be a satisfactory permanent 
employee. 

We expect, for permanent employees, to rate once or twice 
per year 




NEW DEVELOPMENT FOR FIRE MOTOR DRIVER 
EXAMINATION | 

’ WILLIAM E. TRVOQ, JH. 

Kaput Ciry Pwtanod De pa menu, Eimu City, Mlumirl 

The development of a new performance testing procedure 
for, Fue Motor Driven followed some extensive research by the 
Personnel Department of the Cicy merit system of Kansas City, 
Missouri This examination is based upon a standardized teat 
used by the United Stated Army Engineers to examine all moeor 
vehicle drivers, and was developed by Amos E Neyhart, Ad¬ 
ministrative Head, Institute of Public Safety of Pennsylvania 
State College and Consultant to the American Automobile 
Association, The Army examination consisted of a short 
Written exercise followed by tests for visual sciiity, field of 
vision, depth perception, and the applicant’s reaction to simu¬ 
lated traffic situations The Army performance procedure in¬ 
cludes a dosed area and a general road teat, with an appraisal 
of the general driving characteristics of tha applicant. The 
driver’s performance teat developed by the Kansas City Per¬ 
sonnel Department is based pnmanly on the closed area pro¬ 
cedure used by the Army Engineers 

Five tasks were selected (1) Driving on a straight line- 
forward and backward, which consisted of lining up the front 
and rear left wheels on a painted 100-foot line, and driving at 
normal speed, to keep both front and tear wheels on the lure 
for the entire distance (2) Gauging space whin steering in 
dose limits. This is a timed teat m which the contestant has 
to make a 90-degrec turn going forward, and then to baek up, 
making the same turn (3) Stopping the car smoothly m 40 
feet while gpuig 20 miles an hour Lines are painted on the 
street as a guide to begin slowing down and to stop. (4) Stop¬ 
ping the car with the front wheel exactly on a cross painted on 
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the street This procedure is repeated for the 
and also far the rear wheel and bumper.' The 
penalized doubly for going beyond the line. (5 
car against a curb in a regulation parallel parking 

A road test through regular traffic conditions include* 
appraisal of the applicant’s making a tight- and n left-hand 
turn, of his starting from a standstill while on an upgrade m a 
finally, of his handling of the car and of his own reactions’ and 
emotional status in meeting traffic aituatumB A regular tour- 
jng car was used Tor this first test, but there is no doubt that 
fire department apparatus would providu a much more dis¬ 
criminating result It was found that generally the rouruit 
car proved easy for evciyone to maneuver through the vinous 
operations 

One of the heat developments of the Neyhart testing pm- 
cedure is the detailed scoring sheet devised for the use of the 
examiner Under each task are outlined several items upon 
which the contestant is to be graded, e,g. 

Sample Score Sheet 

III Stop ping smoothly in 40 feet at 20 miles an hour—Front Dunnei 
• Driving Forward through Stanchion*, 

1 Moves gear shift lever to another position without dashing 

2, Keeps an even speed—20 M P H 

3 Moves vehicle continuously—no stops 

4 Steers with certainty—no sudden jerks 

5 Does not it op between stanchions 

6. Does not hit right—left stanchions 

7. Does not race engine, 

8 Does not stall engine, 

9 Stops vehicle smoothly. v 

10 Stops front kuinpsr snort—over line 

A marc accurate and discriminating grade is made possible by 
this system, and the driver may lator see the points of the exer¬ 
cise that he failed to perform correctly. The examiner checks 
those point* missed as the operation is being completed and 
scores each exercise before going to tha next It is believed that 
this has proved a much more objective test than others, and by 
the use of the detailed scoring sheet, the standards outlining the 


front bumper 
contestant u 
) Parking the 
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coune, and lines drawn in the street to guide the contestant, 
the result depends lets on the judgment of the examiner and 
provides a positive, equal standard for all applicants. It leaves 
little room for the driver to dispute a point, as there can be no 
doubt when he hits or knocks down a standard, 

The equipment uied for this teat is an important factor, 
both for the applicant and the examiner Standards painted 
white and topped with red flags are used to outline the differ¬ 
ent courses, supplemented by four-inch white lines painted on 
the street at a guide to the driver The talks for this test may 
all be lai^ out m an ordinary street 

The examination for Fire Motor Driver is divided into four 
sections, a written examination, performance test, service rat¬ 
ing, and rating of experience and training. Tlie written and 
performance sections am both given a valuation of 30%, with 
the service rating and raring of experience and training each 
being valued it 20% of the total, The service rating is also a 
new development and was used for the first time in this exami- 
mtiorL Therehasbeen no further opportunity to determine the 
validity of this new procedure, except for the correlations which 
have been computed. Following are the correlations between 
the vaimua sections of the test I Performance and the total 
score,+J8; performance and written,+.02; written and total 
score,+3lj performance and service rating, - 17,* The low 
correlation between the performance test scores »nd written 
■cores is regarded as particularly desirable, since the two sec¬ 
tions are testing for different qualities. 

The scoring and grading system for the performance test is 
baaed on a taw score of 100. The test is divided into ten tasks, 
each bne having a valuation of 10, Points are deducted for 
each Item of the operation that was performed incorrectly, A 
■distribution table, based on rfie score*! u made up, and a post¬ 
ing p3int is set, A conversion table u computed on the bssia 
of the 30% valuation ret for the performance teat The total 
raw score is then converted and added to the reit of the score 
(or the enure examination. 


* film ihfa h ib bit wc of *« ■■mce ridni on as amtaita, wi hid 


The Army Engineers and the Fire Department worked hi 
full cooperation with the Personnel Department in developing 
the examination. With some further refinements, it is believed 
that this will constitute a highly valid and successful testing 
procedure 
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purpose. Dittcnwia Tor making the teat nnlenala, nrecne bMtracuuu for 
u—mkni, l form for recording pcnominn, and norma naiad on the retofda of 
no nibjtru in oven. The author report* that "coefficKno of nliabdii/ deter- 
-u*i by repeiluto th* ihu an all over J>8 or 99 " Of the no pull boo for whkh 
_gnia are gtran, S3 per cent were definitely ngbt eyed on the bn lay, 24 per not 
mi definitely left eyed, 7 par cent Impartial-eyed, and rha remainder thawed left- 
rjti tendency or njfat-eycd tendency BJtik S /ay 
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J undent woiklni runt. PtmaiUi norm* tare been anbluhed teparerdy for 
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Pknjcil Sckaee Coraprehemion, Anthraatie Raronlu, Verbal Comprabenalofl, led 
Uachinlnl Curoptibenalofl, representing tditivdy in dependent aipecti of tcchoKil 
pofoirtnanc#. Mdlih & Joy, __ 


HcJdnitr, K. J “The Refaremihip bitwaeo the Centroid Method and Speannaii'a 
MctM." Jourmi of EdutstknoJ Psychology. XXXV (1944), 347-3i2 
Thla article polnli out the relaileoahlp httween the centroid method of obtaining 
furor meffideou and Spearman'i 1914 method, n wdl u the rdatnothm between 
■In centroid method andSprerrain’t theorem nn the condition of ram* Itliihcwn 
(hit d communalliia ire liuerted in the diemoali, Spcirmer'i method lor oblalnmi 
i factor coefficient n the tame u the centroid pitthod Simflarly, the method of 
CHtejanoo of luma, into with the cumroupalitita Included, ■ oqurvalent tn the 
couniid method, it bnne d anon* t rated that "cmtrend coeffiritna o/ may thoa bo 
itardid aa conbattocu between a v»ruble lj and the total (or avenge) d the 
nnablca projected on the coaunon-ficmr ipact." L. Bailhiltl 


Confer, Homer "SlmpJJyuii the Scvneta Trehnutue of the Berorcuiet Peremwlity 
lormtory " Joins] of ApplUd PryrUofy, SOCVUI (1944), 412-417 
Five almpiified keya were u*cd u replace the Utual aenring technique of the 
Bmnuitr Pttiamsluy fnusnioryr I, AIL ordinal valuta of 3 to -3 were iitwicd a 
value d 0,4 aid above became 1, -4 and below became-1 II Origuitl vafua of 
in-] became 0, 3 and above beam 1. -3 and below became -1 III “f 
■ I inured and all “Yn" and “No" Valua having a difference of tevta 

' Edited by Forrat A. Kmgabury 


344 BDtfCATTONAL AND PSYCHOLOGICAL MEASUREMENT 

(reduced to five oa icila B5—11 or more were paired. Pnaitira til a re became I and 
negative valuta bream*-1 IV “I" retpoma were ignored and ill paiitive mnB 
were coasted ai I d there wea ■ difference of all (reduced “o five On Kilt BJ-1) or 
nwe 'be a a* -1 and "No" an*we ft. V All poeitm rupoiuei were coiuited 

“ I Zero and negatives were woored. After the new anna were derived. Pearson 
correlation* were worked for each idle with itaulta from each key Seventeen M the 
rarreUtnni or the Bra keya with oniinal percentile ranki on the four acaln range 
from 82 to 91 Probable erron range from *.011 to t 032 

It a concluded that the beat new key lor each tint could be uud for quck loci 
ikm of eatreme caeee, the rmddle ifry per cent contldered "normal," and the ifty par 
tent at the two caimnn reacond with the regular Damreuur km. The tuw keya 
effect a living of imu than 90 per cent In (coring tune. Mtrios R Cronei 

Lund in, R W “A PreUmlnary Report on Some New Tot* of Muncal Ability” 
/ouraeJ oIAnltid P/ydolory XXVJII (1944). 19J-39S. 

Tha dree riba i battery of 1 mu dev bid to mature directly and objectively 
muetnl abaitlu, lamed rather than 011111 . The tern—recorded roonnl itamt oa 
■inch “eamr' or differenl' 1 jodjmenu mre requretad for isurvil duermunauon, 
mdodie tnmpo*it»p, htnotonc rrurpniton, rndodt lequencei and hannnnw **• 

n tt*-weio given to 2 rpupi of tubjecu 60 do tic auJenti from DtPiaw and 
ana Uolranlnm and ilm uud to r» from ondcrgraduaic dauet in piychakgy at 
the Inter The total tertrea thawed 1 fallibility of 72 ( hut the retialnliira of the 
lepania uan won not to high. The only ernenon for valtdatam wn the author 1 
epecully euDitructcd gnphic rating tale by whkh the prof tuna of nuiu gnded 
their arudcaa. Pindug a eigruEcant difference between rha 2 aroupi, the author 
bWwvea the lota d~ -- l ---*■ — 1 —- 
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McCarthy, D "A Study d the Rtlbbdicy d the Gnodraouch Drawing Tret d 
Intel Uftoe*." y os real 0 / Pryskclon, XVIII (1944), 201-216. 
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■pier, to 386 third and fourth in de chldrea Each tnt wat teufed three tuna, 
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over, although the conutcncy on two tetta taken at ddftrenz tiroet and uared by 
one puson wu AI, there waa a duerepancy of at Itau obi yr»r In 41 7 per (rat m 
the earn. Th« odd even reliability computed with the Spearman Brown prophecy 
formula waa 89. The ruuln, doTtootiritlng both tha cubjectivny d tcoring and 
tha vanabdnv 16 performance over a abort interval, ahow tha need for caution is 
Dtmg tha Idle for Bodmdoal dugootu, L. ffouidlfaL 
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MeUnae, M A “A Fit tonal Study of Picture Tata for Young Children." Bntvh 
Journal 0} Psychology, XXXV (1944), 9-16 
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lactor promt In ffia ampfeta battery but not ui the picture-tan battery lion a. a 
()) ■ apace facmr, appearing in romc of the picture taau, which abroad a l« differ- 
enca, evtdtnt ody fat tha boyt L BoulMtl 


Oddi, C. W "The Scoring of Continuity or Reamngcment Two." /vs real of 
idweiioadPiyehdon, XXXV (1944), 3S2-35& j 

Tata in which the taag u to rank up to am ft etna in iam« deaignittd order cat 
In eared quickly and in ■ theoretkaJIriouiid manner by uauig a table prrscalsd at 
an aid |n ahonenlitf the rekulatfona. Tha table 1 * divided into accimn* which eiubla 
the icorer m panLue with ntuiia aaora lor poorer than chance arrenRemant, in give 
top aoorea to all amngsiientt peginvdy correlated with the correct order, or ro 
aeon by the better, hue longer, y method whkh require* that difference* between 
ngrai rank* and popil'a raapaore* b* iquared and iumaiad. Bduk S Joy 
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deni Penoanw Work bald at tha lluivenlty of California, Lot Angela, Summer 
Sen km, 1944 PnWuhrd by Wetteru Penonnd Service Peiadena, Calif onua, 
1944 

Tbta M u 20-page pamphlet prepared by a comm litre at the Instmita It aima 
Co promt tha dmaott pooua of view lit a lent* d thort a rode*. & G, Wdluoiton 
it "Leader of Iho Inannire u emphiena the need for a well planped tfllegmed Per* 
umnd program. The ilndBcaDce d “the cantnbumni of the Hudenu iheouelva 
to that own development" a ducoaed by Jcafo E, Gibfga, "Dean uf Wanm." Hm 
"P cnaanel Admnunncor," Karl W On chi ok. In reviewing the 
at the metllnp brurge out maoy of lha pnotlnl coot deration* 1 
cnmadiiu after rha war F T Perkou, at "Firaboloanr," 
the fiet ihar many diverte sgcneire are ulcerated w the pr« 

iugiau that the faculty d cdlege* ibould be brought mu ..._. 

to 1 greater extent. Finally, Ruih P McLain, reprewntmg tha “Laymin, 
Unpraoed by the evidaoca that not only will call eat* be faced with mat dan 
u the pairwar yniv, but that the new prof eta km of penonnd work wdl be of t 
-- 1 -*•— J - J - i. BowHJsl. 
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Rabin, A. I "Teat Conuaivy and Vanaum In tha Manally IB " Journo ( of 
Osirol PiychoLp, XXXI (1944), 331-339 

The WsskjUr-BsUnmd fnidforeare Scots/ wen wad on 60 Now Hampahjre State 
Hoepital adult pi trend to determine the rdaliro tnt-retm cowuuocy of mental 
paiiaan u wdl ■■ tha lenililvhy of iha mtuurmc mimimeoL Tha remit* nvoalad 
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ptycheoca are wry otunble. Tha coefficient aim Indicated a Wah degree of rtju* 
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SwdWogy,XXVnT0944). 388-J92, 
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bath a apllt-haff sad a leu rttaat lludy The correct»d 
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Plana mmmpleto, Annrwnctmaiti will be mitU littl ' ’ * 1 

Thu natanberahip al A-CP-A. will be drodrtned ebfct fuxthsr 


i di^iUj ^Aa ' 

LscBe Brawn, who la w»h the Aratriew Red Craa.lt now mimed to mere- 
rtbnal work In rat cm ten. She wnu* thit ne tipccc ■ her job u to plan toon T 7 
to pouita of historical and cultural Intanec m aornhem id central Italy [_j 

Karl Cowiicry, Atasehte Bcjutrw, Stanford UnWerAj, died In 
Do Cowdcry had bun 1 ra unbar of AC-PA. for mwj(yi * * 

President, as Editor of um Yearbook, and aa Chatrmanlof 


ucua. Hh doth la a gtnt lou to the cryanmtkm 1 

Fotrart Kith pitrlck ha been appobted annuitant 0/1 
ior the Dcturrmrot of State. 
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